NxFilter Tutorial
1. Getting started

2. Blacklist and domain categorization

3. Authentication

4. GUI overview

6. Working with agent

7. NxCloud

8. Customization or rebranding of NxFilter and its clients

9. NxClassifier

10. NxFilter as a DNS server

11. Misc

12. FAQ
System requirements
- Windows, Linux, FreeBSD or other OS having Java(JRE) 1.7 or higher installed.
- 512 MB RAM.
- 4 GB available disk space.
- UDP/53, TCP/80, TCP/443 ports.

* You can run NxFilter with lesser hardware when you have a small number of users but we recommend you to have more than 1 GB of system memory and 40 GB of disk space especially when you have more than 1,000 users.

* At default NxFilter uses up to 512 MB of system memory. This might not be enough for a bigger site. You can increase the limit of the memory allocation to NxFilter in its startup script. In '/nxfilter/bin/startup.sh' modify the value of '-Xmx512m'.

- Go index -
Install NxFilter on Windows
We provide a Windows installer. When you download and run 'nxfilter-x.x.x.exe' you will have the following screen.

After you follow several steps on the installer, it will try to create a Windows service for NxFilter. If you see the following message you have NxFilter successfully installed.

To access the admin GUI, start your browser and type 'http://localhost/admin' into the address bar. Or if you created a desktop icon during the installation process you can click it. If you see the following login screen your NxFilter is up and running. The initial login name and password is 'admin' and 'admin'.

- Go index -
Install NxFilter on Ubuntu Linux
We have a 'deb' package for installing NxFilter on Ubuntu Linux. To install it, install Java first. Download the package using 'wget', and then install it using 'dpkg'. Then start it from the upstart script which is installed with the package.

sudo apt-get install openjdk-8-jre
wget http://www.nxfilter.org/download/nxfilter-3.1.6.deb
sudo dpkg -i nxfilter-3.1.6.deb
sudo start nxfilter

To access the admin GUI, start your browser and type 'http://your-nxfilter-ip/admin' into the address bar. If you install it on '192.168.0.100' type 'http://192.168.0.100/admin'. The initial admin name and password is 'admin' and 'admin'.

When you update NxFilter using 'deb' package and if you update it to v3.1.7 use the following commands,

sudo stop nxfilter
sudo dpkg -i nxfilter-3.1.7.deb
sudo start nxfilter

Your Ubuntu system might be using Systemd instead of Upstart. We install Systemd script as well. You need to use the following commands to start and stop NxFilter service.

sudo systemctl enable nxfilter.service
sudo systemctl start nxfilter.service

- Go index -
Install NxFilter on other Linux
When you install NxFilter on Linux system in general,
- You need to have root permission.
- Make sure that your system has Java 1.7 or higher installed.
- You can start NxFilter as a daemon use '-d' option with 'startup.sh'.

1. Download 'nxfilter-x.x.x.zip' file from www.nxfilter.org.

2. Extract the zip file into '/nxfilter'.

3. Go to '/nxfilter/bin' and run 'chmod +x *.sh'.

4. Run 'startup.sh'.

5. To access the admin GUI, start your browser and type 'http://your-nxfilter-ip/admin' into the address bar. If you install it on '192.168.0.100' type 'http://192.168.0.100/admin'. The initial admin name and password is 'admin' and 'admin'.

* You might want to start NxFilter automatically at your system startup. On our Linux system we have '/nxfilter/bin/startup.sh -d' in '/etc/rc.local' script. You need to use '-d' option for running NxFilter as a daemon.

- Go index -
Install NxFilter on Windows manually
This is about how to install NxFilter on Windows manually using a 'zip' package. You still can make it a Windows service with a batch script included in the package.

1. Download 'nxfilter-x.x.x.zip' file.

2. Extract the zip file into 'c:/nxfilter'.

3. Go to 'c:/nxfilter/bin''.

4. Run 'startup.bat'.

* If you want to install NxFilter as a Windows service run 'c:/nxfilter/bin/instsvc.bat'. It will create 'NxFilter' service. When you uninstall it run 'c:/nxfilter/bin/unstsvc.bat'.

* To run NxFilter as a service 'net start NxFilter'. To stop it 'net stop NxFilter'.

* Use 'net start NxCloud' and 'net stop NxCloud' for NxCloud.

- Go index -
Updating NxFilter
We provide a Windows installer and a 'deb' package for installaing and updating NxFilter. While it is convenient sometimes you have to do it with a 'zip' package. When you update NxFilter using a 'zip' package,

1. Download 'nxfilter-x.x.x.zip' file.

2. Stop NxFilter.

2. Extract the zip file into the directory NxFilter installed.

4. Start NxFilter.

- Go index -
Start and stop NxFilter
There are several utility scripts for NxFilter in '/nxfilter/bin' directory.

- To start NxFilter : startup.sh
- To stop NxFilter : shutdown.sh
- To see if it is running : ping.sh

On Windows use '.bat' files instead of '.sh' files.

* When you run it as a Windows service use 'net start NxFilter' to start and 'net stop NxFilter' to stop.

* Use 'net start NxCloud' and 'net stop NxCloud' for NxCloud.

- Go index -
Client DNS setup
After you install NxFilter you want to monitor and filter Internet activity in your network. To monitor and filter Internet activity you need to make NxFilter to be the only DNS server for your network.

The simplest way of setting up a DNS server for your users would be modifying the network setup on OS level like the above. But you don't want to set up all the PC in your network one by one. So the best way would be using DHCP server. You just need to modify DNS server address on your DHCP server setup and then your users will be using NxFilter as their DNS server.

If you have a firewall you can force your users to use NxFilter as their DNS server by blocking outgoing traffic on UDP/53, TCP/53 port. Now NxFilter became the only DNS server your users can use.

- Go index -
Active Directory integration
NxFilter supports Active Directory integration but some people find it hard to understand. So we want to explain what is Active Directory integration for NxFilter and when to use it and how to implement it at conceptual level.

What is Active Directory integration

One of the reasons why people want to integrate NxFilter into Active Directory is that they want to apply filtering policies based on Active Directory user and group. They also don't want to have their users going through any extra login step to use the Internet except when they login to their own PC. So for NxFilter 'Active Directory integration' means using the same user account from your Active Directory to differentiate users and having single sign-on with your Active Directory.

User importation

Now we know what is Active Directory integration and why we need it. But how to do that? On NxFilter the first thing you need to do for implementing Active Directory integration is to import the users and groups from Active Directory. It means you need to let NxFilter be aware of your users and groups. You can do that on 'User & Group > Active Directory'.

After you import your users and groups, your users will be able to use their Active Directory credentials on NxFilter's login-page. So we already achieved Active Directory integration to a certain level.

Single sign-on with Active Directory

We can say that we achieved Active Directory integration as your users can use their Active Directory credentials on NxFilter's login-page. However your users don't want to go through the login-page so the next thing you need to do is implementing single sign-on with your Active Directory. To impelment single sign-on you need to use an agent program working with NxFilter. We have several agents. NxLogon, NxMapper, NxClient, NxUpdate, NxBlock. You can use just one of them or mix and match them to complement each other.

* For more information, read single sign-on or agent related parts of this tutorial.

MS DNS server and NxFilter

When you deploy NxFilter in an Active Directory environment you might be worrying about the possibility of breaking the integrity of Active Directory as NxFilter is a DNS server and the role of a DNS server in Active Directory is very important. But we don't disable or replace the existing Active Directory DNS server. Our approach is to work with the existing Active Directory DNS server in cooperation. So you have to maintain your existing MS DNS server even though you use NxFilter as the DNS server for your network.

1. Where to install it
Some people try to install NxFilter on their domain controller. But you already have another DNS server there. It is your MS DNS server. It would be better to install it on the other system to avoid of having a port collision problem.

2. Dynamic host update
MS DNS server in Active Directory does a lot of things. It lets the hosts in Active Directory know the location of resources using SRV records. And it maintains a DNS zone for every hosts. It does dynamic host IP update when you change an IP address of a system. To keep all these things working NxFilter bypasses the internal DNS queries for Active Directory domain to MS DNS server automatically. It assumes that you have your MS DNS server on the DC you imported your users from.

3. Which upstream server for NxFilter
You might have a question about which DNS server you should use as an upstream server for NxFilter because you already have a DNS server that is your MS DNS server. You can use any DNS server as an upstream DNS server for your NxFilter including your MS DNS server. NxFilter still forwards your Active Directory internal DNS queries to your MS DNS server. So you can use whichever DNS server you think the best.

4. Manual setup for MS DNS server
After you import Active Directory users and groups, NxFilter tries to work with your MS DNS server automatically based on your Active Directory importation setup but sometimes you want to have a different settings for your MS DNS server. Or you might want to have a redundancy for your MS DNS server. In that case, you can do all these things on the edit page of your Active Directory setup. For having redundancy, you can add multiple DNS servers separated by commas.

* You might need to allow 'Nonsecure Dynamic Update' on your MS DNS zone properties for NxFilter to update the IP addresses of the hosts in the MS DNS zone.

An examplary deployment scenario

Lastly, we will give you an examplary deployment scenario. Suppose we are in a company environment. Many Windows PCs and some Macbooks and recently we bought several Chromebooks. And people bring their own iPhone and Android phones. And plus we have several Linux servers for our own website and file sharing. There are some mobile workers using company laptops. Some are using Windows and some are using Macbooks. And you want to filter all of them whether they are inside office or outside office with their Active Directory accounts.

The first thing you need to do is to set up Active Directory user and group importation. And then use NxLogon for these Windows PCs. But NxLogon doesn't work with Macbooks. For these Macbooks you can use NxMapper. You just need to install it on your domain controller.

And then you want to deal with these mobile workers. You can install NxClient on their laptops. NxClient is basically a remote filtering agent for NxFilter but they will try to do single sign-on when they are in a local Active Directory. There are Windows and Mac versions for NxClient.

For Chromebook, you can try NxBlock. It is a Chrome extension and you can use it as a remote filtering agent or single sign-on agent for Active Directory.

For your servers you better not to filter them and set them up with static IP addresses and use another DNS server for them. You don't need to block anything from them normally.

For your iPhone and Android phones, just let them go through NxFilter's login-page.

- Go index -
When NxFilter not starting
When you find your NxFilter not starting, the first thing you need to do is checking '/nxfilter/log/nxfilter.log' file. You can find some information about the cause of the problem. The other things you might want to check out would be the port collision problem and Java installation. NxFilter uses UDP/53, TCP/80, TCP/443. This means NxFilter itself is a DNS server and a webserver. So if you have another DNS server or webserver running on the same system NxFilter will not start.

About the Java installation, if you use NxFilter's Windows installer, in most cases you will not have the problem but if you install NxFilter manually or if you start it manually not using Windows service you might have some Java related problems. To avoid of having this kind of problem there should be Java installed on the system and you need to have the proper environment variables for Java.

If your are on Windows system having properly configured Java, you will see this kind of message on command line when you type 'java'.

On Windows system you can set these environment variables.

JAVA_HOME = C:\Program Files\Java\jre7
PATH = %JAVA_HOME%\bin;C:\bin

If it is on Linux, NxFilter will try to find 'java' in '/usr/bin' first and then '/usr/local/bin' so if you don't have 'java' in these directories you need to modify the script files in '/nxfilter/bin' directory or you need to include the path into the environment variables for your system.

To set up 'PATH' system variable for Java you can follow the instruction from the link below.

    - http://java.com/en/download/help/path.xml

- Go index -
What is a blacklist?
A blacklist is a database of categorized domains. This is an essential part for a DNS filter for blocking websites by categories. NxFilter supports the following blacklists.

1. Jahaslist
Jahaslist is the default blacklist option for NxFilter. It supports dynamic classification with by NxClassifier. NxClassifier is the integrated auto-classification engine for NxFilter.

For more about NxClassifier and how to grow Jahaslist, read NxClassifier section.

2. Shallalist
Free for non-commercial use only. It is maintained on www.shallalist.de.

3. Komodia
It has more than 30 million domains classified and does dynamic classification. Many companies are are using Komodia DB for their commercial products. NxFilter uses its cloud option so it doesn't require import or update.

- Go index -
Using Jahaslist, Komodia
Jahaslist and Komodia blacklist options are commercial. You can buy the license on our homepage.

* We have a discount option for non-profit organization. When you want to get the discount contact us first using 'support @ nxfiter.org'.

- Go index -
Reclassification on blacklist
You can add domains directly into system categories. It works like the domains added into custom categories. Even if you have the same domain classified differently in your blacklist your custom classification overrides it. So the effect of it is immediate. No need to report it back to blacklist provider and wait to see it reflected.

There are two ways of reclassification. One is to add domains on 'Category > System' and the other one is using the popup reclassification form by clicking a domain on 'Logging > Request'.

- Go index -
Updating Shallalist
NxFilter provides an auto-update script for Shallalist. To update Shallalist stop NxFilter and run '/nxfilter/bin/update_sh.sh'. Depending on Internet speed it may take several minutes to finish the whole process.

If you need to update it manually download http://www.shallalist.de/Downloads/shallalist.tar.gz and extract it into '/nxfilter/shallalist1/BL' then run 'update_sh.sh /nxfilter/shallalist1/BL' command.

cd /nxfilter/bin
update_sh.sh /nxfilter/shallalist1/BL

- Go index -
NxFilter and authentication
NxFilter provides several authentication methods including single sign-on with Active Directory integration.

Why authentication

When you install NxFilter first time you only have one policy and it applies to everybody in your network. But what if you are working for a school as a systems administrator and you want to apply a policy based on user and group. For students, a stricter policy and for teachers, a bit lenient policy. Now you need to differentiate users. That's when you need to enable authentication.

Which authentication

NxFilter supports several ways of authentication. You can choose one of them or mix and match some of them.

1. IP based authentication
This is the simplest form of authentication. When you use a static IP address for your client PC this might be the best choice. Just associate the IP address of the client PC to the user you create on NxFilter GUI. You also can associate an IP range to a user.

* Many people try to use IP based authentication without enabling authentication on 'Config > Setup'. But IP based authentication is still a method of authentication so you must enable it first.

2. Password based authentication
When you enable authentication NxFilter blocks any user trying to access the Internet with its login-page unless they already logged-in or having IP address associated to them. To go through the login-page your users need to enter their passwords. You can set a password for each user on NxFilter GUI.

3. LDAP based authentication
If you have OpenLDAP or Active Directory your users can go through the login-page using their LDAP credentials. To use this feature you need to import your users from your LDAP server first.

4. Login token based authentication
NxFilter has a special concept called 'Login Token'. This is used for remote user authentication or filtering. This login token is being created for each user when you create or import users. You use this login token to differentiate users for remote user filtering with NxClient and NxBlock or dynamic IP update with NxUpdate.

5. Single sign-on against Active Directory
Many people want to filter their users transparently. Or you don't want to show any login prompt to your users. NxFilter provides Active Directory integration. Once you implement it your users don't need to go through NxFilter's login-page and your users will be appeared on NxFilter GUI with their Active Directory usernames and groups.

- Go index -
Single sign-on with Active Directory using NxLogon
When you have Active Directory you want to have single sign-on against it and not showing any extra login prompt to your users. For this we have an agent program that is NxLogon. When you run NxLogon on a user PC it creates and refreshes a login session for the user on NxFilter.

However you don't want to install this program on every PC in your network. So we use a logon script on GPO(Group Policy Object). This logon script is being executed whenever a user logon to Active Directory and launches NxLogon on each user's PC.

* If you want to have single sign-on against Active Directory you firstly need to import users and groups from your Active Directory. To import users and groups read, GUI - User

* NxLogon uses TCP/19002 port to talk to NxFilter.

You can follow these steps to launch NxLogon from GPO.

1. Download nxlogon-x.x.zip package from www.nxfilter.org.

2. Modify IP address in 'nxlogon.bat' to point NxFilter. If you use clustering you can add multiple server IP addresses separated by spaces.

3. Open 'Administrative Tools > Active Directory Users and Computers' on your DC.

4. Open 'Group Policy' tab in properties of your Active Directory Domain.

    

5. Click 'Edit' button and then go to 'User configuration > Windows Settings > Scripts (Logon/Logoff)'.

    

6. Click 'Logon' and click 'Add' and then click 'Browse' button. You will see 'Logon' directory to select file. Copy your 'nxlogon.bat' and 'nxlogon.exe' from NxLogon package into 'Logon' directory. You can drag and drop the files into the directory.

7. Select 'nxlogon.bat' which you copied into 'Logon' directory as a logon script to add.

    

8. Now every time a user logon to his/her system 'logon.bat' will be executed and it will launch 'nxlogon.exe'. You can check the process running on Windows task manager.

    

* If you want to remove login session immediately after user logout use 'nxlogoff.bat' as a logoff script in GPO.

- Go index -
Single sign-on with Active Directory using NxMapper
While using NxLogon is still the best solution for Active Directory single sign-on but some people find it difficult to set up all these GPO and logon script. So we offer an easier way of implementing single sign-on against Active Directory. When you install NxMapper on your domain controller it will grab username and IP address pair when a user logon and creates a login session on NxFilter.

* If you want to have single sign-on against Active Directory you firstly need to import users and groups from your Active Directory. To import users and groups read, GUI - User

Install and run NxMapper

We offer a Windows installer for NxMapper. It will install NxMapper as a Windows service. After you install it, you will have its setup program running.

* NxMapper needs to be installed on a domain controller.

* You can add multiple IP addresses separated by commas if you run a cluster of NxFilter.

* NxMapper uses TCP/19002 port to talk to NxFilter.

After you modify the config values, test your setup first and then start it.

Differences from using NxLogon

Although it is a lot easier to be compared to using NxLogon, NxMapper also has its own limit.

The login session created by NxMapper can be expired. While NxLogon refreshes its login session on evey minutes, NxMapper creates or refreshes a login session only when there is a user activity on a domain controller. Once the session expired your users will be redirected to NxFilter's login-page. To avoid of having this kind of problem you can increase the session timeout value on 'Config > Setup > Block and Authentication > Login session TTL'.

* It can be expired as NxMapper doesn't refresh it. But as long as there is an activity from a user NxFilter refreshes the login session. It usually expires when a user returned to his/her desk from a long break.

Terminal server exclusion

When we use NxMapper we might have a problem with Windows terminal server. If there are multiple users on one system the IP address of the system will be associated to the user whose action detected lastly by NxMapper. It means your users can be appeared on NxFilter with a different username. To prevent having this kind of problem the best solution would be creating an IP based user for your terminal server.

- Go index -
Single sign-on with Active Directory, OpenLDAP using NxClient
NxClient is basically a remote user filtering client for mobile workers with their own laptop. But you can use it for single sign-on against Active Directory or OpenLDAP. One good thing is that since there is Mac OS version of NxClient you can have single sign-on from Mac OS.

If you already tried to use NxClient you already know that single sign-on using NxClient is possible with its 'Login Token' concept. But with this approach one problem is that it is almost impossible to set up several hundreds of NxClient installations with their own 'Login Token'.

So we provide a way of running NxClient on a local network without setting up different login token to each client PC. What you need to do is to install NxClient using a common login token for all the client PC. Then when it starts it will look for its server that is NxFilter on the local network and if it finds one it will try to create a login session for the current logged-in user or console username.

* For NxClient being able to detect a local NxFilter, you have to use NxFilter as the DNS server for your client PC.

* If you use NxClient only for single sign-on you can install it without a login token and a server IP.

To find out more about NxClient read this, NxClient and remote user filtering

- Go index -
Custom login script for single sign-on
Currently NxFilter supports single sign-on with Active Directory. However some people want more than that. For example, you might want to have single sign-on with your OpenLDAP.

NxFilter supports an API set for creating login session through HTTP protocol. You need to write your custom login script to call some webpage on NxFilter's built-in webserver. And then your users don't need to go through NxFilter's login-page.

We have example code on '/nxfilter/example/login_user.jsp'. Initially the access of the page is restricted to localhost only for security reason but you can edit the JSP page to allow the calls from your local network.

You can call the webpage this way.

http://192.168.0.100/example/login_user.jsp?ip=192.168.0.100&uname=john

As you see there are two parameters being passed. One is the IP address of your user and the other one is the associated username. The username should be imported or created on NxFilter side already.

One thing you need to consider when you write your own login script is that it might be better to call the webpage periodically. There is a session timeout concept in NxFilter. If there is no activity from a logged-in user for certain amount of time the login session will be expired. So if you don't want to show your users NxFilter's login-page, you would need to refresh the login session periodically.

There are three methods of UserLoginDao class for custom login script.

- create_ip_session(String ip, String uname) : Creating a login session with an IP and username.
- delete_ip_session(String ip) : Deleting a login session with an IP.
- find_user(String ip) : You can find a logged-in username by its associated IP address.

All the example JSP pages are in '/nxfilter/webapps/example' directory.

- Go index -
The order of authentication methods
NxFilter supports multiple authentication methods at the same time. But there is a sequential order for these authentication methods. You need to understand the order of authentication methods to build your filtering setup.

This is the order of authentication methods.

1. Single IP association
Single IP association comes first so that you can exclude some systems from the IP range association or allow some users to login without any login procedure.

2. IP session
IP session is the login session being created and maintained on NxFilter by its single sign-on agent or login-page. This comes at second.

3. IP range association
When you need to allow an anonymous user to access the Internet without any login process you create an IP range user to cover whole your network. But you still want to differentiate users by single IP association or the login session. So the IP range association comes at last.

We have 'Most specific IP range comes first' rule for ordering IP range users. If there are overlapped IP ranges the smaller IP range will be applied before the others.

- Go index -
GUI - Config
These are mostly system configuration parameters for NxFilter.
Config > Setup > Block and Authentication

- Block Redirection IP
This is the IP address of NxFilter itself. If there is a blocked DNS request, it will be redirected to this IP address. It is supposed to be populated automatically during the installation process.

* You can add multiple block redirection IP addresses separated by commas for redundancy.

- External Redirection IP
When you use a remote filtering agent you might need to use a different 'Block Redirection IP' for the remote filtering agent since it is outside of your network. If you leave this one empty NxFilter will use 'Block Redirection IP' for redirecting the remote filtering agent.

- IPv6 Redirection IP
You need to set this up when you use NxFilter as your IPv6 network DNS server.

- Enable Authentication
This option is required when you use any authentication methods including IP based authentication. After you enable this option, any unauthenticated user will be redirected to NxFilter's login-page. As a result your users will be forced to login to use the Internet.

- Login Domain
You can access NxFilter's login-page using a domain defined here.

- Logout Domain
You can clear out a user login session using a domain defined here.

- Login Session TTL
NxFilter keeps a login session after a user login. But this login session needs to be expired eventually. It is especially required when there is a shared PC by several users. If a user doesn't make any DNS request for the specified amount of time defined here, his/her login session expires and the user needs to login again.

Config > Setup > Syslog

NxFilter supports Syslog exportation of its log data. You can build your own reporting system with this feature or you can monitor all the logging in real-time way.

- Syslog Host
The host IP address to which you want to send syslog data.

- Export Blocked Only
With this option NxFilter sends the log data of blocked DNS request only.

- Enable Remote Logging
Enable Syslog exportation.

Config > Setup > NetFlow

NxFilter supports bandwidth control. This is possible by importing NetFlow data.
For more detail read this, Bandwidth control with NxFilter

- Router IP
The IP address of a device sending NetFlow data to NxFilter.

- Listen Port
The UDP port number of NetFlow collector.

- Run Collector
Run NetFlow collector. After change this option you need to restart NxFilter.

Config > Setup > Misc

- Admin Domain
You can access the admin GUI using the domain you set up. For example, if you use 'admin.nxfilter.org' as your admin domain you can access your admin GUI by typing 'http://admin.nxfilter.org/admin' into your browser address bar.

* This only works when you use NxFilter as your DNS server. Otherwise you need to register your admin domain to your own DNS server.

- Bypass Microsoft Update
You don't want to block Microsoft update with your filtering. Enabling this option means bypassing 'micfosoft.com' and 'windowsupdate.com' and their subdomains.

- Logging Retention Period
If you keep your log data too long it will use your disk space a lot. You can set how long NxFilter keeps its log data here.

- SSL Only to Admin GUI
When you want to allow only HTTPS access to the admin GUI enable this option. Once you enable this option you will be redirected to the SSL port automatically even if you use HTTP.

- Auto Backup
NxFilter makes a backup file for its config into '/nxfilter/backup' directory on '01:00' everyday. The name of the backup file starts with 'auto-' prefix. You can have up to 30 backups.

- Agent Policy Update Period
NxFilter provides several agents programs for application control and remote user filtering. These agents fetch their policies periodically. You can set up the policy update period for them here.

Config > Admin

You can change admin name and password for GUI login here.

* 'Client Password' is for remote filtering agent setup. We use it to access NxBlock setup page.

Config > Alert

NxFilter sends an alert email for recent blocking or clustering node down incident. If you want to send an alert email to 'admin @ nxfilter.org' from 'alert200 @ gmail.com' on every 15 minutes then the setup would look like the followings.

- Admin email : admin @ nxfilter.org
- SMTP host : smtp:gmail.com
- SMTP host : 465
- SMTP SSL : on
- SMTP user : alert200
- SMTP password : ********
- Alert period : Every 15 minutes

Config > Allowed IP

NxFilter has IP based access restriction function for its DNS, GUI, login redirection. You may need to use this feature when you put your NxFilter on a public IP address. You can make whitelist/blacklist way of ACL here.

Config > Backup

You can make a backup for the config DB of NxFilter manually. The backup files will be created into '/nxfilter/backup' directory.

Config > Block Page

This is the setup for custom block-page, login-page, welcome-page. When you edit your block-page you can use the following variables populated by NxFilter for making your block-page more informative.

- #{domain} : Blocked domain
- #{reason} : Reason for block
- #{user} : Logged-in username
- #{group} : Groups of the logged-in user
- #{policy} : The applied policy
- #{category} : Categories or the blocked domain

Config > Cluster

NxFilter has a built-in clustering. You can make your NxFilter to be a master node or a slave node in a cluster. After you change the values in cluster setup you need to restart your NxFilter to apply the new settings.

- Go index -
GUI - DNS
NxFilter is basically a DNS server with filtering ability. These are DNS service related configuration parameters.
DNS > Setup > DNS Setup

- Upstream DNS server
NxFilter works as a forwarding DNS server. You need to have at least one upstream DNS server for NxFilter.

- Upstream DNS Query Timeout
Timeout for a DNS query to your upstream DNS server.

- Upstream DNS Load Balance
Load balancing option for your upstream DNS servers.

- Max Client Cache TTL
You can modify the TTL value in a DNS response from NxFilter. If you set the value to '60' NxFilter modifies the DNS cache TTL to '60' if the TTL is bigger than 60.

0 - Don't touch it.
60 - Don't touch it if it's smaller than 60 and make it '60' if it's bigger than 60.

We introduced this function to minimize the effect from the client cache. However if you have more than 1,000 users you would better turn this function off for better performance.

- Response Cache Size
NxFilter has its own cache for DNS query result from its upstream server. Generally speaking, the bigger cache would be better for the performance. Currently the default size is 200,000 and it is enough for most sites.

DNS > Setup > Local DNS

- Local DNS Server
When you have a local DNS server for resolving your local domain add your local DNS server IP address here. You can add multiple IP addresses separated by commas for redundancy.

- Local Domain
When you have a domain which you want to forward to your local DNS server add the domain here. You can add multiple domains separated by commas.

* Don't use '*' or any wildcard for a local domain.

- Local DNS Query Timeout
Timeout for a DNS query to your local DNS server.

- Upstream DNS Load Balance
Load balancing option for your local DNS servers.

- Use Local DNS
Enable local DNS.

* If you set up a local DNS server for you local domain, all the DNS queries for your local domain will be bypassed from authentication, filtering and logging.

DNS > Setup > Dynamic DNS

NxFilter supports dynamic DNS service. For howto, read Dynamic DNS server part of this tutorial.
DNS > Zone File

When you use NxFilter as an authoritative DNS server you would need to set up a zone file. We use the same format as BIND zone file. To find out more read Authoritative DNS server part of this tutorial.

DNS > Redirection

Domain to IP or domain to domain redirection is possible with NxFilter. It works like a custom DNS record.

DNS > Zone Transfer

In some situation you need to import a DNS zone from another DNS server. Once you add a zone-transfer setup here NxFilter imports the DNS zone on every minutes using IXFR protocol.

- Go index -
GUI - User
You can create or import users and groups here. NxFilter supports user importation from Active Directory and OpenLDAP.

Creating a user

You can create a user on 'User & Group > User'. There are 3 types of users in NxFilter

1. IP user
It has an associated IP address or IP range and will be authenticated based on IP address.

2. Password user
If you set a password for a user it becomes a password user. You can use the password on the login-page of NxFilter.

3. LDAP user
When you import users from your LDAP servers or Active Directory they become LDAP users. They can use LDAP or Active Directory user credentials on NxFilter's login-page.

Properties of a user

- Password : The password for login through NxFilter's login-page.
- Work-time Policy : The policy being applied when it is not in a free-time.
- Free-time Policy : The policy being applied during a free-time. You can define a free-time on 'Policy & Rule > Free Time'.
- Expiration Date : The expiration date for a user account.
- Login Token : The token for remote user filtering or remote user authentication. It is created when a user created or imported.

Testing a user

When you have an LDAP imported user you may have multiple groups and policies for a user. As a result it becomes difficult to figure out which policy a user fall into. To find out which is the 'Applied Policy' for a user, use 'TEST' button on the user list. It fetches the state of a user from NxFilter in real-time way.

* You can use this test view to find out how much quota or bandwidth consumed by a user or to reset quota or bandwidth for a user.

Creating a group

You can create a group on 'User & Group > Group'. After you create a group you can set up a policy for the group by editing its properties. You also can assign members to the group.

Importing users and groups from Active Directory or OpenLDAP

You can import users and groups from Active Directory on 'User & Group > Active Directory'. For example, if you have your Active Directory with the following setup.

- Domain controller : 192.168.0.100
- Domain : nxfilter.local
- Admin username : Administrator

Then you can create an Active Directory importation setup with the following details.

- Host : 192.168.0.100
- Admin : Administrator@nxfilter.local
- Password : your-password
- Base DN : cn=users,dc=nxfilter,dc=local
- Domain : nxfilter.local

After having an Active Directory importation setup you can import users and groups with 'IMPORT' button. You also can set up a periodical import by selecting an auto-sync interval.

* If you want to find out if there is a connectivity issue between NxFilter and your domain controller, use 'TEST' button.

- Go index -
GUI - Policy
With NxFilter you can have multiple filtering policies based on user and group.

Creating a policy

When you install NxFilter, there is only one policy that is 'Default'. This policy will be applied to everybody if you don't make any change on NxFilter setup. If you want to apply a different policy for a specific user or group you need to create another policy and enable authentication.

Editing a policy

After you create a policy you can modify its properties.

- Priority Points
If there are multiple policies associated to one user then the policy having the biggest points will be applied.

- Enable Filter
If you disable this option there will be no blocking from the policy.

- Block All
Block everything on policy level.

- Block Unclassified
Block unclassified domains.

- Ad-remove
Block domains in 'Ads' category of Jahaslist with a blank block-page.

* This is useful when you want to remove embedded adverts without showing NxFilter's block-page.

- Max Domain Length
There are some malwares using domain name itself as a message protocol. These domains are abnormally long while the length of most domains are under 30 characters. You can set a limit for the length of a domain to block these abnormal domains. To prevent having false positives NxFilter doesn't apply 'Max Domain Length' against 100,000 well known domains.

- Block Covert Channel
Some malwares or botnets are using DNS protocol as their communication tool. They are using DNS query and response to communicate to each other.

- Block Mailer Worm
Normally you are not supposed to see MX query from your client PC. When NxFilter finds MX type query from your client PC it will be regarded from some malware trying to send emails.

- Block DNS Rebinding
When NxFilter finds a private IP address(192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8) on DNS response packet it will be blocked as DNS rebinding attack.

* If you have your own DNS record with private IP address you need to bypass the domain on whitelist.

- Allow 'A' Record Only
This is the most strict way of filtering malwares and botnets employing DNS protocol as their communication tool. If you are an ordinary office worker you don't need to use any special type of DNS record. With this option NxFilter allows A, AAAA, PTR, CNAME only and the other types of DNS records will be blocked.

- Quota
NxFilter has quota-time feature. You can allow your users to browse some websites for a certain amount of time. You can set the amount of time here.

- Quota All
Apply quota to all domains including unclassified domains.

- Bandwidth Limit
You can set the bandwidth consumtion limit for a user.

This feature requires to import NetFlow data from your router or firewall. For more details read Bandwidth control with NxFilter on this tutorial.

- Safe-search
Enforcing safe-search against Google, Bing, Yahoo and Youtube.

* Safe-search enforcing for Yahoo requires a local proxy agent running on user system.

- Block-time
You can set policy specific block-time.

- Disable Application Control
Disable application control on policy level.

- Disable Proxy Filtering
Disable proxy filtering for on policy level.

- Logging Only
Monitoring user activity without blocking them.

- Blocked Categories
You can block DNS request by categories.

- Quotaed Categories
If you check some categories in 'Quotaed Categories' then your users can access the websites in the categories for the amount of time you specified with 'Quota' above. When a user consumed up his quota his/her DNS requests for those sites will be blocked.

Define a free-time

You can define a global free-time in 'Policy & Rule > Free Time'. If you assign a free-time policy to users it will be applied during the time defined here.

* If the start-time is bigger than the end-time then it will break into 'end-time ~ 24:00' and '00:00 ~ start-time' on the same day.

* You can set a group specific free-time on 'User & Group > Group > EDIT'.

Application Control

NxFilter provides application control through NxClient. For more details read 'Application control with NxClient' part of this tutorial.

Proxy Filtering

NxFilter provides HTTP proxy filtering through NxClient. For more details read 'Proxy filtering with NxClient' part of this tutorial.

- Go index -
GUI - Category
On NxFilter there are system categories and custom categories. System categories are already defined by your blacklist DB. But you can create your own custom categories. You can add domains into these system/custom categories and block domains by these categories.

Currently NxFilter supports several blacklist options. If you want to find out more read 'Blacklist and domain categorization' part of this tutorial.

* If you want to include subdomains into a category use asterisk.

    ex) *.nxfilter.org

* If you want to find out which category a domain fall into, use 'Category > Domain Test'.

- Go index -
GUI - Whitelist
This is for making a whitelist/blacklist by a domain or a keyword here.

- Bypass Authentication : When you need to bypass authentication for some sites use this options.

- Bypass Filtering : When you want to exclude some domains from your filtering use this option.

- Bypass Logging : When you find that there are too many log data for a domain which you are not interested in you can bypass logging for the domain with this option.

- Admin Block : When you want to block some domains without setting up a policy use this option. This option overrides 'Bypass Filtering'.

* You can use an asterisk to include subdomains.

    ex) *.nxfilter.org

- Go index -
GUI - Main, Logging, Report
NxFilter keeps its log data up to 400 days and you can generate a daily, weekly, per-user report based on the log data.
Main

When you login to your admin GUI you will see the dashboard of NxFilter. There are several charts for showing a summary for the last 2 hours. On the bottom of the dashboard you can see 10 recent block logs for the last 12 hours.

* The difference between 'request-sum' and 'request-cnt' is from NxFilter's logging system. To reduce the amount of disk access NxFilter keeps all the log data into its memory space. And then it flushes the data once a minute. If there is a request for the same domain from the same user in a minute it only increases the count for the data. So 'request-sum' means the sum of all the counts and 'request-cnt' means the count for all the unique data.

Logging

You can search user request log with various conditions in 'Logging > Request'. Logging data is being updated once in a minute to reduce the load on DB.

In 'Logging > Signal' you can view the log of the signals from the agents of NxFilter.

In 'Logging > NetFlow' you can monitor NetFlow data imported.

* Use square brackets for the exact matching keyword on log search.

    ex) [nxfilter], [192.168.0.100]

Report

NxFilter generates a daily, weekly, per-user report.

- Go index -
Differences between agents
NxFilter has several agents. Some are for single sign-on with Active Directory. Some are for remote user filtering and dynamic IP update. Some of them can be used for application control and proxy filtering.

1. NxLogon
Single sign-on agent for Active Directory. You can launch it from a logon script on GPO.

2. NxMapper
Another option for single sign-on with Active Directory. Unlike NxLogon you install it on a domain controller.

3. NxClient
Remote user filtering agent of NxFilter. When you have a mobile worker or home worker you can install NxClient on their laptop.

4. NxUpdate
Dynamic IP updater for NxFilter.

5. NxBlock
Remote filtering agent and single sign-on agent for Chromebook.

- Go index -
NxClient and remote user filtering
NxFilter provides a remote user filtering client software that is NxClient. Once you install NxClient on a user system you can filter and monitor the Internet traffic from the user system on your NxFilter GUI centrally regardless of its location.

* You need to open 53 UDP and 80, 443 TCP ports on NxFilter side.

Installation of NxClient

After you install it using NxClient installer you will have its setup program running. There are 'Server IP' and 'Login Token' parameters and you need to set them up with your own.

* On NxFilter every user has a login token. You can find it on 'User & Group > User > Edit'.

* NxClient is a Windows service program. It will start at system startup automatically.

* When you install NxClient or NxUpdate on Mac OS X, read Installing NxClient or NxUpdate on Mac OS X on this tutorial.

After you modify the config value test your setup first and then start it. You can check if it is working by viewing 'Logging > Signal'. There will be signals from NxClient.

* You can add multiple server IP addresses separated by commas if you run a cluster of NxFilter.

* To change the config value run 'C:/Program Files/nxclient/setup.exe'.

Signals of NxClient

When it comes to a remote user filtering the most tricky part would be how to force users to be filtered. Nobody wants to get filtered and they are away from your office. If they use their personal PC then you can not filter them anyway. But when they use a company laptop you still can filter them by installing NxClient on their system.

However what if they uninstall or stop NxClient? NxClient is running as a service and it doesn't provide an uninstaller for 'Add/Remove programs' in control panel. So if your users don't have enough privilege they can't uninstall it.

But sometimes you need to give your users 'Local Administrator' privilege. In that case it's not possible to stop your users from uninstalling NxClient. So we defined several signals with which you can find out what's happening on a user system. NxClient send the following signals.

- START : When NxClient starts it sends START signal to NxFilter.
- STOP : When NxClient stops it sends STOP signal to NxFilter.
- PING : On every 5 minutes NxClient sends PING signal to NxFilter.

You can view these signals on 'Logging > Signal' on NxFilter GUI.

Fail-safe measure for NxClient

NxClient needs to update its pollicy from its server that is NxFilter. When NxClient can't connect to its server it bypasses filtering as your users need to be able to use the Internet anyway. You can specify multiple server IP addresses on its setup for redundacy.

Auto-switch between local filtering and remote filtering

When you use NxClient on your mobile worker's laptop you might have a problem with your filtering policy when they are staying in the office. Your mobile worker might be filtered twice. One from NxClient, one from your local NxFilter. And he/she might be required to go through the login-page of NxFilter.

To address this issue NxClient does auto-switch between local filtering and remote filtering. This means that NxClient can find NxFilter in a local network and when it is on your local network it stops its proxy filtering. Plus, it has its own NxLogon module doing single sign-on in your local network.

* If you don't like this auto-switch behavior you can add 'no_switch = 1' into 'C:/Program Files/nxclient/conf/cfg.properties'.

Uninstalling NxClient

To prevent an accidental uninstallation by your user, NxClient doesn't provide an uninstaller for 'Add/Remove programs' in control panel. When you uninstall NxClient you need to do it manually with the following commands.

- Run 'C:/Program Files/nxclient/bin/unstsvc.bat'.
- Delete 'C:/Program Files/nxclient' folder.

Silent install

Some people want to install NxClient on multiple PCs using GPO or PDQ deployment. For this, we have the silent install option for NxClient.

For silent install,

/silent : Runs the installer in silent mode (The progress window is displayed).
/verysilent : Very silent mode. No windows are displayed.

And you can specify 'Server IP' and 'Login Token',

/server=192.168.0.100
/token=2P1WQ6VF

This is the final form of the command.

    nxclient-6.0-win.exe /silent /server=192.168.0.102 /token=2P1WQ6VF

* You can build your own MSI package using MSP wrapper from http://www.exemsi.com.

* When you install Java silently as a prerequisite for NxClient it might not be starting. This is mostly because when you install Java silently it doesn't set 'PATH' environment variable for itself.

- Go index -
NxUpdate and dynamic IP update
When you have a client system using dynamic IP address and you want to associate its IP address to a specific user you can install NxUpdate on the system. It will update the user's associated IP address on NxFilter.

NxUpdate has basically the same structure with NxClient. You can install it in the same way as NxClient.

* It sends START, STOP and IPUPDATE signals.

* NxUpdate can be working as a dynamic DNS client for NxFilter.

Writing your own NxUpdate

We use the standard DNS protocol for communication between NxFilter and NxUpdate. This means you can write your own NxUpdate as long as you can run 'nslookup' or if you can send a DNS query.

* NxFilter v3.3.0 or later and NxUpdate v7.0 or later required for the DNS protocol based communication.

If you send an IP update query against NxFilter from your Windows CMD using nslookup,

nslookup GKSYEJYG.ipupdate.signal.nxfilter.org. 192.168.0.100

'GKSYEJYG' is a login-token of a user and 'ipupdate.signal.nxfilter.org' is the special domain for 'IPUPDATE' signal. '192.168.0.100' is the IP address of your NxFilter.

We use the following signals.

- start.signal.nxfilter.org : 'START' signal.
- stop.signal.nxfilter.org : 'STOP' signal.
- ipupdate.signal.nxfilter.org : 'IPUPDATE' signal.

* You need to add a login-token of a user to these signals for user identification.

When we send these signals we can have two kinds of responses as a DNS response from NxFilter.

- 127.100.100.1 : Error.
- 127.100.100.100 : Success.

You don't need to send 'START' or 'STOP' signal if you want to go simple. Sending 'IPUPDATE' would be enough.

- Go index -
Application control with NxClient
NxFilter supports application control with NxClient. You can block an unwanted program by setting up your application control policy on NxFilter GUI centrally and you can find out who tried to run the blocked programs on your log view.

How it works

After you define your application control policy on 'Policy & Rule > Application Control' NxClient retrieves the application control policy periodically.

* You can adjust the policy update period by changing the value for 'Agent Policy Update Period' on 'Config > Setup'.

Supported options

1. Block by port scanning
NxClient detects UltraSurf and Tor processes by port scanning. This means even if your users change the process name or run them from a USB stick you can find and block them.

2. Block by process name
You can block a process running by its name. This works based on keyword matching against process name. You can add the blocked keywords on GUI and If your agent finds the matching process name it will block the process.

Enable application control only for specific users

Basically the application control of NxFilter supposed to be a global policy. When you want to disable it for some user use 'Disable Application Control' option on 'Policy & Rule > Policy > EDIT'.

Logging blocked application

NxFilter is basically a DNS filter so its log data format was designed for showing the allowed/blocked domains. To accommodate the log data about a blocked application NxFilter introduced these domains or rules.

- ultrasurf.port.app : UltraSurf has been blocked by port scanning.
- tor.port.app : Tor has been blocked by port scanning.
- chrome.exe.pname.app : Chrome has been blocked by its process name.
- Skype.title.app : Skype has been blocked by its window title.

* When you enable 'Block UltraSurf' and there is UltraSurf extension for Chrome or other extensions having proxy permission installed NxClient kills Chrome process and sends 'ultrasurf.chrome.app' signal.

Execution Interval

Finding and blocking application may cause some amount of CPU load. If you don't want to cause too much load form your client PC, increase 'Execution Interval' on 'Policy & Rule > Application Control'.

- Go index -
Proxy filtering with NxClient
NxClient has a local web proxy mudule for HTTP protocol level filtering.

How it works

Firstly, define your proxy filtering policy on 'Policy & Rule > Proxy Filtering'. After NxClient started on user system they will filter HTTP traffic by setting up itself as a local proxy server. NxClient retrieves the proxy filtering policy periodically by 'Agent Policy Update Period' on 'Config > Setup'.

Supported options

1. Block HTTPS
You can block all the HTTPS traffic.

2. Block IP Host
Blocking HTTP requests with IP host in URL.

3. Block Other Browser
NxFilter's proxy filtering is being activated through the system proxy settings. Internet Explorer and Chrome are using the system proxy already and many other applications are also using the system proxy. But there are some applications trying to make a direct connection to the Internet. With this option enabled NxClient will block any program making direct HTTP connection to the Internet.

* Currently the proxy filtering supports Internet Explorer, Chrome, Firefox.

* You can allow direct HTTP access to some application using 'Excluded keywords' on 'Policy & Rule > Application'. Basically it is for application control, but it also bypasses 'Other Browser Blocking'.

4. Blocked Keyword in URL
Keyword filtering against URL.

5. IE Proxy Bypass
NxClient bypasses the domains you have on 'Whitelist > Domain' with 'Bypass Filtering' option. But this only applied on HTTP protocol on Windows. When you need to bypass the other protocols than HTTP or the sites using the other ports than TCP/80 add the sites here. They will be appended to the system proxy bypass list on Windows.

Enable proxy filtering only for specific users

The proxy filtering of NxFilter works globally. If you need to disable it for some user check 'Disable Proxy Filtering' option on the 'Policy & Rule > Policy > EDIT'.

Logging

You only get domain level log data. But you will see a detailed block reason like the followings.

Domain: www.google.com
Reason: Blocked by proxy, url_kw=game

- Go index -
Installing NxClient or NxUpdate on Mac OS X
We provide Mac OS installer for NxClient and NxUpdate. You can install them as you do with Windows installer. You set up connection values and do 'Test' and 'Start'.

When you want to run the setup program, you need to run 'setup-mac.sh' script inside the installation directory. If it's NxClient it would be installed into '/Library/nxclient' so you'd need to run the following command.

sudo /Library/nxclient/setup-mac.sh

When you uninstall it,

sudo /Library/nxclient/uninstall-mac.sh

- Go index -
NxBlock for Chromebook
NxBlock is the remote filtering agent for Chromebook. It also can be used as a single sign-on agent in a local network.

* NxBlock has became an open source software since v1.8. You can download its source code from our download page.

Installation of NxBlock

NxBlock is basically a Chrome extension. You can install it from Chrome Web Store. You can download it from the following link.

     - NxBlock download from Chrome Web Store

Filtering policy of NxBlock

NxBlock shares the policy on 'Policy & Rule > Proxy Filtering' with the other proxy filtering agents. It updates its policy on every 120 seconds.

Connection to NxFilter

After you install it, you can see NxBlock on your extension setup panel of Chrome or 'chrome://extensions'. There is 'options' link under NxBlock icon. When you click the icon you will be on NxBlock setup page. You need to set up these parameters.

- Sever IP : The IP address of your NxFilter.
- Login Token : A login token associated to a user on NxFilter.

Once you set up these parameters you can test the connectivity using 'Test' button. And then use 'Save' button to save and reload the new configuration.

Password protection of your setup

You can hide your NxBlock setup page from your users by having password login procedure. Once you set up a password and enable it, the users will be blocked from accessing NxBlock setup page and 'chrome://extension'.

* You can use your client password from NxFilter to access NxBlock setup page once its connection to server is established.

User identification

We use login token and Google account to identify users. Suppose you create a user named 'student' and setup 10 NxBlock with the login token associated to 'student'. If these users don't login to Chrome they will be appeared on NxFilter side as 'student' but if one of them login to Chrome using 'john1234@gmail.com' for example, then he/she will be appeared as 'student_john1234' on NxFilter log view.
Central configuration for mass installation

When you do mass installation for NxBlock the problem is that you don't want to set up its connection parameters one by one. If you just use it as a single sign-on agent in your local network it might be fine without the connection parameters but if you want to use it for remote filtering you must set these parameters.

To solve this problem, we have a way for setting up these values centrally. We use a webpage and Chrome's start page function for this. Simply speaking, you write a webpage containing these config values and then make that webpage to be Chrome's start page on Google admin console. Then everytime your users start their Chrome they will set up themselves with the values.

When you write the webpage you add a meta tag like the followings,

<meta name='nxblock' content='192.168.0.100:HW00IYKW:1'>

We have 3 parameters separated by colons. The first one is NxFilter's IP address and the second one is a login token and the last one is about locking or unlocking Chrome's extension setup page.

On Google admin side,

1. From the main dashboard, go to Device Management > Chrome > User Settings.

2. Select the organizational unit to which you want the settings to apply.

3. Find 'Pages to Load on Startup'.

4. Enter the URL for the web page containing NxBlock configuration meta tag.

5. Click the 'Save Changes' button.

- Go index -
What is NxCloud?
NxCloud is a fully rebrandable multi-tenancy cloud based DNS filter software. It is based on NxFilter and inherited the most of the features of NxFilter. Simply speaking, you can build your own cloud filtering service like OpenDNS.

These are the features only available on NxCloud.

Multi-level admin

If you want to build your own cloud service, one of the essential factors would be being able to create accounts for your customers and the customers need to be able to set up their own policy on their own GUI.

On NxCloud there are 3 kinds of users.

Admin > Operator > User

'Admin' is actually the administrator of NxCloud. It has almost the same GUI as the NxFilter but being an administrator you can create the operator accounts. These operator accounts are for your customers and it is something like a sub-admin on NxCloud. They can create and manage their own users and policies.

Creating an operator

To create an operator you need to login to NxCloud GUI with admin account. On 'Config > operator' you can create an operator. When you create an operator NxCloud creates a default user and a default policy for the operator with the same name.

You can change the number of users and policies the operator can create. This means you can have several levels on your service based on the permission for an operator.

Operator GUI

On NxCloud each operator have their own GUI. If you login to NxCloud GUI with an operator account you will be on the operator mode GUI. It is a bit more restrictive compared to the admin GUI as you only can manipulate the operator specific parameters.

Operator and user

Operators can create their own users and apply a different policy based on user authentication. Users can be authenticated based on IP address or using NxClient.

Operator specific dashboard and report

Dashboard and report of NxCloud is still available on operator GUI.

Operator specific free-time

Each operator can define their own free-time and they can set up a work-time policy and a free-time policy for their user.

Operator specific whitelist and blacklist

You can add operator specific whitelist/blacklist based on domain name. But you still have the global whitelist/blacklist for admin. So you can have more flexibility to deal with these whitelist and blacklist as an admin.

Operator specific alert-email

NxCloud sends an alert email about the blocking incidents to each operator. Operators can setup their email addresses to receive the email and define alert period on 'Config > Alert'.

* You need to set up the global alert email first to send the operator specific alert email. You can set it up on 'Config > Alert'.

Operator specific block-page

Each operator can have their own block-page. If there is no block-page defined by operator NxCloud shows the default block-page by admin.

Authentication over cloud

NxClient still works against NxCloud. This means you can differentiate users behind their router and you can apply a different policy based on user.

Dynamic IP updater

Many of your clients will be using the service from a dynamic IP address. You can use NxUpdate as a dynamic IP updater for your service.

Dynamic DNS association

Some of your users may have a dynamic domain for their own network. You can associate a domain to a user on NxCloud.

Dynamic DNS service

NxCloud supports dynamic DNS service. You can build a dynamic DNS service like DynDNS or No-IP. We use NxUpdate as the client program for this service.

Rebranding or customization of GUI

Its GUI layer is designed for easy customization. The GUI layer is separated from its core part. You just need to modify all the JSP pages in '/nxcloud/webapps' directory. These JSP files have a naming rule corresponding to the GUI menu structure. So it is easy to find which file you need to modify.

- Go index -
Pricing plan for NxCloud
NxCloud does not require any license other than its blacklist license when you use Jahaslist or Komodia blacklist. The minimum amount you need to buy is 500 user license for 1 year. You can have your own customized GUI. You also can customize our agents.

* When you increase NxCloud license user number, you can add it for the remaining period. For example, if you need to add 100 user license 6 months after you start your business then its price would be for the remaining 6 months and after 6 months you renew all the license at once.

* If you want to use the other blacklist option than Jahaslist or Komodia we charge 2 USD per-user, per-year. You need to have a license from your blacklist provider. At the moment we support Zvelo, NetSweeper blacklist options for NxCloud.

- Go index -
Install NxCloud
NxCloud is basically a modified NxFilter. You can install and run NxCloud in the same way as NxFilter.

But unlike NxFilter, after you install it, you can't use it as your DNS server right away. This is because NxCloud is a multi-tenancy program for commercial service. You are not supposed to use it for your internal network. Your clients use it for their network. So you need to create an account for your client first.

On NxCloud there are 3 kinds of users.

Admin > Operator > User

'Admin' is you and an operator is your customer and a user is the user in your customer's network. An admin manages operator accounts and an operator manages the end users and policies. So you need to create an operator first. To create an operator login to NxCloud GUI as admin and then go to 'Operator' menu. You can create an operator there.

When you create an operator there will be a default user and default policy for the operator with the same name as the operator. And the default password for the operator is also the name of the operator. Once you create an operator you can login with the operator account to set up a user for testing.

* You need to associate your IP address to the default user of your first operator to test it.

- Go index -
Differences from running NxFilter
1. Authentication enabled always
You don't want to make your service available to everybody for free. You want to service it to your paid customers only. so the authentication is enabled by default.

2. Login redirection disabled at default
You still can use the password based login with NxCloud but if you use that on a public network you can be a target of DDOS attack. You would better disable it on a public network. When you disable it NxCloud silently drops the DNS request packets from unknown source IP addresses.

3. Magic password for accessing operator GUI
As an administrator of NxCloud sometimes you would need to access operator GUI for technical support purpose. For that reason, NxCloud has one more password for admin. It is called 'magic password'. With this password you can access any operator's GUI. The default magic password is 'magic1023' and you can change the password on 'Config > Admin'.

- Go index -
Business account and home account
When you build a cloud based filtering service one of the problems you have is to find out the exact number of users behind a router. It may be possible when there are some kind of agent installed and running behind router and NxCloud supports several agents for that. However some of your customers don't need to differentiate users and they just want to have one global policy for everybody. It means you don't know how many users they have.

To solve this problem, we limit the request count for a user. Currently one user can make 3,000 requests a day. This is more than enough considering a user makes under 1,000 requests a day according to our statistics so far. However we may have another issue from this request count limit approach. If you have a customer using your service in their home they probably have several Internet accessing devices and have several family members but not wanting to pay for multiple users. In that case this 3,000 daily request limit is too small for them.

To address this issue, we introduced the concept of operator type. There are 2 kinds of operator types on NxCloud. One is 'Business' and the other is 'Home'. Business type operators are nothing special. You can create as many users as you want for them and each of the users has 3,000 request limit. But if it is a home type operator its first user has 7,000 extra request count and that makes 10,000 daily request count limit for the first user. If it has the second user its request count limit becomes 13,000 and that would be more than enough for most home users.

* For home type operator you can create up to 5 users.

- Go index -
Writing your own billing system for NxCloud
When you service NxCloud commercially you want to have an automated billing system. Since all the GUI layer exposed as JSP pages it is not that difficult for you to build your own builling system with NxCloud.

To build your own billing system you need to be able to create, edit an operator which is your client account on your service. You should be able to do these actions on your custom JSP pages. Suppose if you need to create an operator with these properties.

- Name : triton
- Password : triton1234
- Email : tmail0487@yahoo.com
- Max user : 3
- Max user IP : 3
- Max policy : 3
- Max whitelist : 20
- Max free-time : 10

The JSP code would look like the followings.

<%
OperatorData data = new OperatorData();
data.name = ”triton”;
data.passwd = ”triton1234”;
data.email = ”tmail0487@yahoo.com”;
data.max_user = 3;
data.max_user_ip = 3;
data.max_policy = 3;
data.max_whitelist = 20;
data.max_free_time = 10;

OperatorDao dao = new OperatorDao();
dao.insert(data);
%>

If you need to update the properties of an operator.

<%
OperatorDao dao = new OperatorDao();

OperatorData data = dao.select_one_by_name(”triton”);
data.max_user = 5;
data.max_user_ip = 5;
data.max_policy = 5;
dao.update(data);
%>

If you need to suspend an operator.

<%
OperatorDao dao = new OperatorDao();

OperatorData data = dao.select_one_by_name(”triton”);
data.stop_flag = true;
dao.update(data);
%>

* There is a separated section for GUI customization on this tutorial and we also provide Javadoc for building your own custom GUI.

- Go index -
Before you customize NxFilter
* This is for our business partners. We allow customization and rebranding of NxFilter when you have a special license agreement with us.

Now we will talk about how to customize or rebrand NxFilter and its client softwares with your own brand. Firstly, we will show you how to customize its GUI. And then we will talk about the other things you might have interest in.

- Go index -
GUI - Directory structure and naming rule
The GUI layer of NxFilter was designed for easy customization. It is completely separated from its core part. And it has a naming convention corresponding to its menu structure so that you can find the file you need to modify easily. For example, if you want to modify 'Policy & Rule > Free Time' on NxFilter menu the file you need to edit is '/nxfilter/webapps/policy,free_time.jsp'.

* In NxCloud's case, it has operator specific menu. If a JSP file is for operator specific menu then it has 'zop' prefix.

    ex) zop,policy,free_time.jsp

Structure of web application directory

We put all the JSP pages into '/nxfilter/webapps' and we don't use any subdirectory for keeping JSP pages. This is for the simplicity and easy understanding. Everything you need to modify is in '/nxfilter/webapps' directory. It has the following structure.

/nxfilter/webapps
- error
- example
- img
- include
- lib
- WEB-INF

In 'webapps/error' directory we have the error pages for HTTP error codes. If you want to have the error pages for other error codes you can define it on '/webapps/WEB-INF/web.xml'.

* We use HTTP 400 error for special purpose. You shouldn't define any error page for HTTP 400 error.

In 'webapps/example' directory we have some example JSP pages for custom login module.

In 'webapps/img' we keep image files for webpages.

In 'webapps/include' we have common JSP files to be included into the other JSP files. These are for library functions and navigation menus and initialization code for JSP pages.

* '/include/lib.jsp' is a common library file for all JSP files. It has some utility functions for web development and it executes the initialization code for JSP pages and does authentication checking as well.

* We don't include '/include/lib.jsp' directly. It gets included when we include '/include/top.jsp'.

In 'webapps/lib' we have CSS and javascript files.

We have 'WEB-INF' as we use an embedded Tomcat to be NxFilter's built-in webserver.

Separating your customized GUI into another directory

When you customize NxFilter's GUI it is not a good idea to modify the original files directly. You would better keep it for future reference and create another directory under the installaion directory of NxFilter and copy all the files inside '/nxfilter/webapps' into the new directory and then modify these copied files. For this, NxFilter supports 'www_dir' option on '/nxfilter/conf/cfg.properties' file.

So if you have your own custom GUI in '/nxfilter/myweb' directory. You need to add this line into your cfg.properties file.

    www_dir = myweb

Then restart your NxFilter.

- Go index -
GUI - Using Dao and Data classes
On typical web programming, dealing with DB is almost everything. We are using 'Data Access Object' and 'Data Object' for manipulating DB.
Common methods for a data access object

We have some common methods for most data access object classes. For example, on 'policy,policy.jsp' file we use PolicyDao and PolicyData class for manipulating policies. PolicyDao has these methods.

public int select_count() : The number of policies.
public List select_list() : Fetching policies as a list.
public PolicyData select_one(int id) : Fetching one policy by ID column.
public boolean insert(PolicyData data) : Insert a new policy.
public boolean update(PolicyData data) : Update a existing policy.
public boolean delete(int id) : Delete a policy by ID column.

Every policy data has its own unique ID which is a number and we use this ID for finding, updating a policy data.

Insert, delete, update, select data

If we want to modify 'whitelist,domain.jsp' we have to use 'WhitelistDomainDao' and 'WhitelistData' classes.

To insert a new data,

<%
WhitelistDomainDao dao = new WhitelistDomainDao();

WhitelistData data = new WhitelistData();
data.domain = "*.nxfilter.org";
data.bypass_auth = true;
data.bypass_filter = true;

dao.insert(data);
%>

To delete a data when its ID is 12,

<%
WhitelistDomainDao dao = new WhitelistDomainDao();
dao.delete(12);
%>

To select a data when its ID is 12,

<%
WhitelistDomainDao dao = new WhitelistDomainDao();
WhitelistData data = dao.select_one(12);
%>

And to update the selected data,

<%
data.bypass_filter = false;
dao.update(data);
%>

Lastly, to list data.

<%
WhitelistDomainDao dao = new WhitelistDomainDao();
List data_list = dao.select_list();
for(WhitelistData data : data_list){
    out.println(data.domain + "<br>");
}
%>

Accessing data field

Many Java developers are using 'get' and 'set' accessors for encapsulation and for having some additional data processing like validation. But we use public field directly in most cases. For example, you get an instance of UserData and uses its 'name' property like the following codes,

<%
UserData data = new UserDao().select_one(1);
out.println(data.name)
%>

But there are some data classes having methods starting with 'get_'. These methods are mostly about formatting. We have 'ctime' property for 'RequestData' which we use on 'Logging > Request'. If you use it directly you get '201507081415' but when you use its 'get_ctime()' method you get '07/08 14:14'.

- Go index -
GUI - Javadoc for Dao and Data classes
We have Javadoc for 'dao' and 'data' packages.

- Go index -
Templates for email and block-page
NxFilter sends alert emails to its administrator. Mostly it is about access violation for the blocked sites but there are emails about clustering node failure or license violation. We have two email templates for these alert emails.

- /nxfilter/conf/tpl/access_violation.ftl
- /nxfilter/conf/tpl/alert_email.ftl

In '/nxfilter/conf/tpl' directory you also can find the templates for block-page, login-page and welcome-page. These templates are being used when you first install NxFilter to populate its DB or when you click 'RESTORE-DEFAULT' button on 'Config > Block Page' on NxFilter GUI.

- Go index -
Other things you may be interested in
You might want to replace or remove our 'readme.txt' and 'license.txt' with your own files. You can do that but you still need to keep our 'license.txt' file somewhere. We keep all the third party licenses in '/nxfilter/third-party-license.txt' and you also can add our license into that file. About our 'readme.txt', you can remove it or replace it with your own.

There are links to our online tutorial in '/nxfilter/tutorial.html' and '/nxfilter/bin/tutorial.bat' You can remove or replace these files when you make your own package for your customized NxFilter.

The other thing you would need to think about would be icon files. There are two icon files. One for Windows program icon and the other one is for favicon of its admin webpage. You can remove '/nxfilter/nxd.ico' and '/nxfilter/webapps/favicon.ico' or replace them with your own icons.

* You shoud not remove our license file or any third party license.

- Go index -
Making your own install packages for the client softwares
You can make your own packages for our agents that are NxClient, NxUpdate, NxMapper, NxLogon, NxBlock.

For NxLogon, since it is a simple Windows console application without installer you just need to replace several files from the original zip file and make your own zip file for it. You can change its name as well. When you change its name you also need to change the contents of the included batch files but these are all straight forward.

However it is a bit different for NxClient and NxUpdate, NxMapper as these softwares require you to make your own installers for Windows and Mac OS.

Making your own Windows installer

In our case, we use Inno Setup from http://www.jrsoftware.org to build the Windows installers. When you install NxClient, NxUpdate and NxMapper, they will create their own directories inside 'C:\Program Files (x86)' and register them as a Windows service. For example, when you run NxClient installer we copy all the files into 'C:/Program Files (x86)/nxclient' and then we run 'bin/setup.bat' in the installation directory and then at the end of the installation process we run 'bin/instsvc.bat' to register it as a Windows service. When you uninstall it, we run 'bin/unstsvc.bat' to unregister it from Windows service list.

* You can use your own packager but we can provide our Inno Setup script and 'zip' file version of each client when you become one of our business partners.

Making your own Mac OS installer

We use 'Packages' from http://s.sudre.free.fr for building our Mac OS installer. When you run our installer it will create its own directory under '/Library' and copy a 'plist' file into '/Library/LaunchDaemons' to run it as a daemon. and then it runs 'setup-mac.sh' inside its installation directory. When you uninstall it, you need to run 'uninstall-mac.sh' inside the installation directory manually.

* We can provide our Packages script and 'zip' file version of each client when you become one of our business partners.

Changing application name

When you customize our agents the first thing you want to do might be changing the names of our agents. We have 'appname' file for NxClient, NxUpdate and NxMapper. The file is in 'conf' directory under its installation directory. If you want to change the agent name then change the contents of the file. When you build your own installer with your own 'appname' file then these softwares will show the name from the file to your users.

Replacing icon file and default setup value

When you want to replace icon, the icon file is 'nxd.ico' inside the installation directory and it is a merged icon file for 16x16 and 32x32 and 48x48 icon files. At the moment it is only for Windows Installer and setup program.

* For Java version NxClient you need to add one more icon file which is 'nxd16.png'. It's 16x16 PNG file for its setup GUI.

One of the other things our business partners want to do is to change the default connection values to their server especially when they want to use it for their cloud service. You can change these values for 'Server IP' and 'Login Token' in 'conf/cfg.default' file.

Customization of NxBlock

NxBlock is an open source software. You can download its source code from our download page.

Limitation

Building your own installers and changing the names of the client softwares will do what you want to do mostly. But there is something you can't touch or change. We have some internal code having 'nxfilter' signature. This is important as we need to have a unique signature to diffrentiate signals from our agents.

And you don't remove our license or any third party license from the package otherwise that is a license violation. You can have your own license file but you need to keep our license somewhere. All in all it is our software and you just customize it, so it is inevitable to have some limitation.

- Go index -
What is NxClassifier
NxClassifier is the NxFilter's integrated auto-classification engine for Jahaslist. It does dynamic classification against the websites visited by your users base on keyword matching and scoring system. You can define or modify its classification ruleset as you like using your own language.

* NxClassifier requires a valid Jahaslist license.

- Go index -
Why NxClassifier
There are two kinds of classification methods we can use to build a blacklist. One is auto-classification and the other one is human classification. Auto-classification does the job at very fast speed but it is not as accurate as human classification. And human classification costs a lot of money as you need to hire real people. So the best method would be the combination of both or at least having a human verification process for the auto-classification result.

Most of these auto-classification engines do their job based on a keyword matching system with a keyword dictionary or some kind of ruleset. So we need an effective keyword dictionary or ruleset for an auto-classification engine to make a satisfiable result. But we can have different opinions on which keyword belongs to where and how one keyword important to be compared to the others.

Moreover we also can have a different opinion on the classification of a site. For example, 'www.stackoverflow.com ' is a community or 'Forum' site but also can be classified into 'Computer/Technology' category as they are focusing on computer and technologies. And yet they are one of the 'Business/Service' websites. We can classify it into all of them if we allow multi-categories for a site. But in that case, in our filtering business we face another problem. What if we want to block 'Forum' but allowing 'Computer/Technology'? We get an unexpected result.

Another one, at the moment we speak English. But English is not the only language we use in the world. There are numerous non-English websites. So we need the keyword dictionaries for the other languages as well. But who is going to build all these dictionaries for all the other languages? And if we do human classification or verification we need to hire people being able to speak those languages too. So things are getting bigger and complicated. Nobody can afford it. And that's why we have many false positives for these blacklists and we can't be satisfied with any blacklist 100%.

Lastly, no matter how big it is, a blacklist can't cover the whole Internet. Even if you have a huge file to cover the Internet, how many websites in the file could be useful for you? Your users will never visit most of the websites in it as they are not written in your own language.

However with NxClassifier,

1. You build your own ruleset.
This means that you can build your own classification ruleset for your own language. You are the best one for filtering the websites in your own language.

2. We don't allow multi-categorization.
One site can be classified into only one category. No more confusion from multi-categorization.

* The reason for these blacklist providers having multi-categories for one site is not just that they want to reflect the real world ambiguity. They want to use it for the other market something like advertising market. That's why they need to have multi-categorization. But it is no good for us in most cases.

3. You can do recategorization instantly.
No need to make a recategorization request to a blacklist provider and wait for them to accept your request. You can modify your own blacklist as you like.

4. Jahaslist keeps growing.
We only ship a baselist in NxFilter package as we want to make the size of NxFilter package small. It covers the major sites only. But with the aide of dynamic classification by NxClassifier it keeps growing and they are only for the actual websites your users are visiting.

5. You can share the result with others.
Once you build your own blacklist using NxClassifier you can share it with others. Not only the blacklist. You can share your classification ruleset.

- Go index -
Jahaslist and its directory structure

Jahaslist is the name of the blacklist which is being grown by NxClassifier. Jahaslist is being activated when you install NxFilter. We have '/nxfilter/jahaslist' directory for it and we have several files in it.

- categories.txt : Jahaslist categories are defined here.

- baselist.txt : Even if we can build our own blacklist with NxClassifier we still need all these major websites classified. Otherwise you will see too many unclassified websites before NxClassifier classify enough number of domains.

- ruleset.txt : This is the default classification ruleset for NxFilter.

* baselist.txt and ruleset.txt will be imported into NxFilter DB when you start NxFilter first time.

* We keep everything except the classification ruleset in a separate DB file. It is '/nxfilter/db/jahaslist.h3.db'. When you want to reinitialize Jahaslist for some reason you just need to delete this file and restart NxFilter.

- Go index -
GUI for NxClassifier

We have 'NxClassifier' top menu on GUI. It has the following submenus.

Setup > Classifier Setup

- DNS Test Timeout : NxClassifier only classifies the existing domains. So it does DNS testing first when it needs to classify a domain.

- HTTP Connection Timeout : After DNS testing, now it needs to download a webpage to analyze. This is the connection timeout value for HTTP connection.

- HTTP Read Timeout : This is the data read timeout value after you have an HTTP connection.

* If you make these timeout values too big you might have a performance degrading for NxClassifier.

- Classified Data Retention Days : NxClassifier makes the classification result log for the recently classified websites. NxClassifier doesn't do the classification if it is already classified or if it is in the classification log without an error.

- Keep HTML Text : NxClassifier can extracts the text from a main page of website and keep it for reclassification. But this requires more disk space so you can decide to keep the text or not.

- Disable Domain Pattern Dic : NxFilter has a domain pattern based calssification process. If a domain can be classified by this domain pattern dictionary NxClassifier doesn't try to classify the domain.

- Disable Classification : You can disable the classification by NxClassifier.

Setup > Mass Import

You use this when you want to import a ruleset file or Jahaslist file exported from another system. It doesn't delete the existing data. But it overwrites the existing data if it is the same one.

Ruleset

You can define your own ruleset here. Or modify the default ruleset. You can export the ruleset and share it with others.

Classified

This is the classification result log by NxClassifier. It will show yout the recently classified domains and how they got classified or unclasssified. Based on this classification result you can improve your classification ruleset.

* With 'VIEW' button you can view the details of the log and with 'TEST' button you do the actual classification process for a domain with your current ruleset.

* If you want to apply a new classification ruleset against the already classified sites use 'RECLASSIFY-ALL' button.

Excluded

We exclude the domains making certain errors during the classification process. For example, if we have 403 response from a website we don't need to try to classify it as we can't access the website. Or if we get an image file or some other type of file instead of a text or HTML file we will exclude it.

* Since we don't delete these excluded domains if you want to have NxClassifier trying to classify an excluded domain you would need to delete it from the list first.

Jahaslist

You can view the contents of Jahaslist and modify it directly here. But we don't recommend you to do the reclassification here unless it is a mass importation of domains. We keep Jahaslist in a separated DB file and NxFilter doesn't do auto-backup for it. So you would better use 'Category > System' for reclassification as it is in the main config DB.

* When you do the reclassification on 'Logging > Request' or 'NxClassifier > Classified' your reclassification data goes into 'Category > System'.

* When you export Jahaslist, NxFilter merges your custom classified domains from 'Category > System' into Jahaslist and then export the merged result into a file.

Test Run

After you add your own classification rules you want to see the result. You can do a test run for your classification ruleset against a website here.

* 'Test Run' doesn't do actual classification. If you want to classify a domain you need to make a query for the domain against NxFilter.

- Go index -
Understanding NxClassifier workflow
You need to understand the workflow of NxClassifier before you can build an effective classification ruleset for you.

NxClassifier itself is a multi-threaded program integrated into NxFilter. When NxFilter finds an unclassified domain requested, it adds the domain into the process queue of NxClassifier. NxClassifier does DNS test to see if the domain actually exists and then it tries to see if there is a website for the domain. If there is a website for the domain NxClassifier downloads its main page and parses its title, description and the text.

Once NxClassifier gets the details of a website then it runs the data through its classification ruleset. While running through its ruleset, if it finds an exactly matched rule it stops there and classify the domain to the associated category to the rule. Otherwise it adds up the points from the matching rules and classify the domain into the category which has the biggest score.

- Go index -
What is a classification rule
Now we need to understand how to make a classification rule. A classification rule consists of the following factors.

- Keyword : Matching keyword. In reality it is a regular expression.

- Target : You can apply your keyword against the domain, title, description and text of a website.

* We get the title, description and text of a website from its main page.

- Points : You can set a different points to a rule by its importance. The minimum points to be classified is 100 and the maximum points is 1,000.

- Category : Associated category for a rule.

* When you want to have an 'Exclude Keyword' for a category, set minus points for the keyword. For example, you associate 'movie' keyword to 'Entertainment' category but you don't want to classify the websites you can download movies to 'Entertainment'. Then you associate 'download' keyword to 'Entertainment' with minus points.

- Go index -
Performance tuning for NxClassifier
If you run it on a big site having many users you might notice that NxClassifier's classification speed can't follow up all the unclassified domains your users visited. This is because we designed NxClassifer to be an optional service for NxFilter. You run NxFilter basically for filtering not for classifying domains. So we don't want to have NxClassifier consuming all the system resources. So we run only two worker threads for NxClassifier at default.

However some of you want to increase its performance. In that case you can increase the number of threads by setting up a value for 'classifier_num' parameter on '/nxfilter/conf/cfg.properties' file.

classifier_num = 8

And when you have a cluster for NxFilter, NxClassifier also gets clustered. With this feature you also can have a dedicated node for classification. You run 16 threads for classification on one node and you set 'classifier_num' to zero on all the other nodes.

The other thing you need to think about would be the optimization of your classification ruleset. If you have a bigger ruleset NxClassifier needs to consume more CPU power.

- Go index -
Remote repository for Jahaslist
NxFilter does auto-update for Jahaslist on the background. It downloads the update files from the Internet and updates its Jahaslist. You can build your own repository for this remote update. This is useful for people wanting to share their blacklist or making a business on their blacklist.

1. The process of the auto-update.
When you add a URL of an update file from the Internet on 'NxClassifier > Setup > Jahaslist Repository' NxFilter downloads the file and updates its Jahaslist. The auto-update process will be running at the startup time and once a day nightly. Since the auto-update process is running as a separated thread, it doesn't block NxFilter main process.

* You can add multiple repository URLs separated by newlines.

2. The format of an update file.
Basically it needs to be a text file with '.txt' extension. But you also can upload a zipped tarball with '.tgz' or '.tar.gz' extensions. In the zipped tarball you need to have a text file. So if there is a minor update you would better go with a text file and if is a major update it would better be a zipped tarball. Inside the file you can add a domain and a category number concatenated with a comma like the followings,

google.com,46
yahoo.com,46
nxfilter.org,9

Actually this format is being used for importing and exporting Jahaslist on GUI. This means you can share the exported file from your Jahaslist by uploading it into a website.

* If you use a text file without archiving, you can add 100,000 domains at the maximum into the file.

* When you make 'recatlist.tgz' file for 'recatlist.txt' file, use the following command.

    ex) tar cvzf recatlist.tgz recatlist.txt

3. The version of an update file.
You don't want to download the same file over and over again. So we have a versioning system for the update files. When you have your udpate file on the following URL,

http://www.nxfilter.org/test/recatlist.txt

NxFilter tries to find a version file on this URL,

http://www.nxfilter.org/test/recatlist.ver

In the version file you need to have a number. When it is greater than the version number NxFilter previously downloaded it downloads the update file and updates its Jahaslist and if the number is less or equal, NxFilter ignores the update file.

* The version number can not be less than 1.

- Go index -
NxFilter as a DNS server
NxFilter is basically a forwarding, caching DNS server with filtering ability. You also can use it as an authoritative DNS server. It also supports dynamic DNS service.

- Go index -
Forwarding DNS server
When you install NxFilter first time it is already ready for working as a forwarding DNS server. It uses Google DNS servers as its upstream DNS servers at default. You can change the upstream servers on 'DNS > Setup'.

- Go index -
Caching DNS server
NxFilter has its own cache for the DNS response from its upstream server. This means when you use a public or ISP DNS server as a DNS server for your network NxFilter can boost up your network speed by reducing the traffic to the DNS server outside your local network. This is because once a DNS response from an upstream server has been cached then your users will get the response from NxFilter not from the upstream server. You will not have a latency problem between your local network and the DNS server on the Internet.

- Go index -
Authoritative DNS server
NxFilter can be working as an authoritative DNS server.

1. Zone File
We use the same form of zone file as BIND. You create a zone file for a domain on 'DNS > Zone File'. And then you can add your hosts into the DNS zone by editing it on GUI.

2. To put it on the Internet
Since NxFilter is a DNS filter with authentication, when you use it as an autoritative DNS server there are several things you would need to think about.

- Authentication
You must enable authentication especially when you put NxFilter on the Internet to avoid of being a target of DDOS attack. But the problem is that if you enable authetication, these anonymous users querying your domain will be redirected to the login-page of NxFilter. To allow the anonymous DNS query against your domain, you need to bypass authentication for your domain.

- Filtering
NxFilter is a DNS filter so your domain might be blocked by NxFilter for some reason. This will lead to the failure of resolving the domain you want to service. To avoid of having this kind of problem, it might be better to bypass filtering for your domain.

- Too many log
You could have too many log data for your domain as a result of DDOS attack. In that case, you would better bypass logging for your domain.

* You can set up whitelist for your domain with some bypass options but you also can do that using the bypass options of a zone file.

* On NxCloud you have 'Public Service' option instead of bypass options. In reality this is a combination of 'Bypass Filtering' and 'Bypass Logging' as there is no 'Bypass Authentication' option for NxCloud. You need to set this option to service your DNS zone on the Internet when you use NxCloud.

3. Clustering
When you build a cluster of NxFilter your slave nodes will be working as an authoritative DNS server with the settings from the master node. You don't need to set up a secondary DNS server for redundancy. It is already clustered.

- Go index -
Dynamic DNS service
NxFilter supports dynamic DNS. You can build a 'DynDNS' like service with NxFilter if you want.

To service dynamic DNS you need to have a domain for the service on 'DNS > Setup > Dynamic Domain' and then enable the service. If you want to service it publicly or on the Internet, you will have to have a authoritative DNS zone on 'DNS > Zone File' for your dynamic DNS domain.

Once you set up everything on the server side, you need to install a dynamic DNS client on your client system. We use NxUpdate for this. On NxUpdate you need to set up a server IP that is the IP address of your NxFilter and a login token associated to a username. The associated username will be the hostname.

For example, if you have 'example.com' as your dynamic DNS domain and you install NxUpdate on a client system with a login token which is associated to 'myhost' user. Then once your NxUpdate started working you can access the system using 'myhost.example.com'.

* Dynamic DNS service requires to enable authentication on 'Config > Setup'.

* You can view the list of dynamic domains being serviced on 'DNS > Dynamic Domain'.

* On NxCloud we have 'Logging > Dynamic DNS' for admin to monitor the activities related to your dynamic domain.

- Go index -
To avoid of having DDOS attack
When you put NxFilter on the Internet you might be under DDOS attack. Once you are under DDOS attack or the other kinds of DNS attack you could have a massive traffic to your NxFilter. Your NxFilter can't handle all the traffic and eventually it will look like almost dead and you will get the error logs about 'Queue full'.

To avoid of having this kind of problem, the best thing is to hide your DNS server or not responding to these attackers with a valid DNS response. To hide your NxFilter as a DNS server from these attackers you can enable authentication firstly. Being unknown to NxFilter, these attackers will get your NxFilter redirection IP as a DNS response.

But still they may think there is a DNS server to attack as they get a response anyway. To hide it from these attackers completely we need to drop the packets from these anonymous users silently. For this purpose, you can set 'Allowed IP for Login Redirection' on 'Config > Allowed IP' and NxFilter will drop the packets from these attackers.

- Go index -
Clustering with NxFilter
NxFilter has built-in clustering for load balancing and fail-safe. Once you have a master node you can add up to 4 slave nodes to your cluster. All the slave nodes in your cluster share the setup from the master node. So you can control everything on your master node.

Creating a cluster

To create a cluster, the first thing you need to do is to make a master node. On 'Config > Cluster' you can make one of your NxFilter installations to be the master node. And then you can add the other NxFilter installations as the slave nodes to your mater node. You need to restart NxFilter after change the cluster setup.

* Clustering requires 19002, 19003, 19004 ports on TCP opened on the master node.

Starting clustered NxFilter

When you start NxFilter cluster you need to start the master node first and then you can start the slave nodes. This is because your slave nodes need to download the initial setup from the master node when they start.

Load balancing and fail-safe

One good thing about a DNS filter is that there is already a way of load balancing and fail-safe existing. Make your master node to be the primary DNS server and your slave node to be the secondary DNS server in your network. Then you get the load balancing and fail-safe.

* If you want to have these load balancing and fail-safe for your block-page and login-page or policy update for NxFilter's agents you need to set multiple block redirection IP addresses separated by commas on 'Config > Setup'.

When a cluster node down

When a slave node down the other slave nodes and master node will not be affected. The rest of the nodes will be working normally. But when your master node down you lose filtering but DNS lookup still working. When the master node restored your slave nodes re-establish the connection to the master node automatically.

If you set up the alert email on 'Config > Alert' you will receive an email for a cluster node down.

Access control for slave nodes

If you add all your slave node IP addresses into 'Config > Cluster' any attempt to join a slave node from an unknown source IP address will be blocked.

Monitoring slave node state

You can view connection state from your slave nodes on 'Config > Cluster'. Once you set up your cluster then your slave nodes will be appeared with the last contact time on the page. It is also showing each node's request, block, user, client-ip count information. These counter information will be set to 0 on midnight or when you restart NxFilter.

Session sharing between cluster nodes

We use TCP ports for sharing data between cluster nodes. The one of the data we share is the login session so that you don't need to login twice to master and slave node. And we share quota-time and bandwidth consumption data as well. But this could be a reason for performance degrading when you have busy servers as it increase the amount of communication between nodes.

If you don't want to share these data you can disable authentication and not to use quota-time and bandwidth control. But you may want to have authentication even if you need to login twice. And in reality, this login session sharing is only for the login-page. If you don't use password login you are not going to have any problem. NxLogon and NxMapper, NxClient can talk to multiple NxFilter servers. And IP based authentication works fine without session sharing.

To disable session sharing while you keeping authentication enabled, add this line into '/nxfilter/conf/cfg.propertie'.

    no_share_session = 1

* You need to set up 'no_share_session' option on all nodes.

- Go index -
Bandwidth control with NxFilter
NxFilter supports per-user based bandwidth control using NetFlow data. The idea is simple. NxFilter associates NetFlow data from a router to its user login session based on IP address and if there is a user consumed bandwidth over the limit you set, NxFilter blocks all the DNS requests from the user.

Good thing is that this is not just about HTTP traffic. Since NxFilter uses NetFlow data you can monitor and block other protocols including HTTP, FTP, IM, Skype, Torrent and any other protocols working on TCP/UDP.

To enable the bandwidth control you need to have a router or firewall supporting NetFlow version 5 in your network and you need to make them sending NetFlow data to NxFilter. And then run NxFilter's built-in NetFlow collector on 'Config > Setup > NetFlow'. After that you can set up a bandwidth limit on a policy.

There are several rules for NxFilter to import NetFlow data. Firstly, one of the source or destination IP addresses of a NetFlow data should be associated to an IP address of a logged-in user on NxFilter. Secondly, NxFilter ignores the internal traffic. This means one of the source or destination IP addresses needs to be a public IP address. This is because you are only interested in inboud or outbound traffic to the Internet. And lastly, NxFilter keeps only TCP/UDP data.

* Currently NxFilter supports NetFlow v5 only.

- Go index -
Detecting and preventing malware/botnet activity with NxFilter
One of the features of NxFilter is to be able to detect and block malware/botnet activity by analyzing DNS packets. In reality malwares and botnets are another form of network clients or server programs. This means that they are also heavily relying on DNS protocol to find their masters or peers to communicate with or the victims to attack.

For example, if you have a spambot in your network the spambot will make a lot of DNS queries for MX records of the target domains to send emails. But normally your client PC doesn't need to make MX queries unless they have a mail server running on it.

Another example would be the botnets using the 'TXT' record or other DNS records as their communication tool. These are the real world examples of malwares using DNS protocol as their communication tool.

ex1) Trojan.Spachanel was using SPF record.
ex2) W32.Morto was using TXT record.

The other method we can think of would be detecting the abnormal length of domains. When we tested top 100,000 domains by Alexa all the domains except 142 domains were shorter than 30 characters. But there are abnormal domains trying to look like some URL of a target website. This is an example from www.phishtank.com which is trying to look like a webpage of www.ebay.co.uk but actually it i s a phishing domain.

ex1) cgi.ebay.co.uk-item-css.ebay-motors.session.id-sj3mzbasf3k12z581668115.login-wpadmin-sw.buyitnow.sign-in.secure-process657943sddh53zix34235hj65rj.xml.config-page.overview.buyer-protection-jsp.wpcs.spiridus-magic.org

So detecting botnet/malware by analyzing DNS packet would be one of the effective techniques we can think of. NxFilter provides these blocking options on its policy setup.

- Max Domain Length
- Block Covert Channel
- Block Mailer Worm
- Block DNS Rebinding
- Allow 'A' Record Only

But you can say that the most effective way of preventing malware/botnet in your network would be allowing only 'A' record query from your client PC. In most cases your client PC doesn't need to make a DNS query for any other record than 'A', 'AAAA', 'PTR' , 'CNAME'.

- Go index -
Removing embedded adverts in webpages
There are webpages having embedded adverts from other domains. One of the problems for blocking these adverts with NxFilter would be having a mangled webpage as a result of blocking. Your block-page replaces the embedded adverts.

To avoid of having this kind of problem, there are two ways of removing embedded adverts with NxFilter. One is using a special category in 'Category > Custom' which is called 'ad-remove'. If you add a domain into this category and block the category somewhere, NxFilter blocks the domain with a blank block-page.

The other method is to block it using the 'Ad-remove' option on a policy. With this option NxFilter blocks 'Ads' category of Jahaslist.

* After you add a domain into 'ad-remove' category you need to block the domain on whitelist or policy otherwise it will not be blocked.

- Go index -
Syslog exportation
NxFilter provides Syslog exportation function. The exported data is a character string separated by '|'. For example, if you have the following Syslog data,

NXFILTER|2013-01-28 10:53:23|Y|www.bbc.co.uk|pwuser|192.168.0.101|admin|news|Blocked by admin|33

It can be parsed into these values,

- Prefix : NXFILTER
- Date : '2013-01-28 10:53:23'
- Block yes/no : Y
- Domain : www.bbc.co.uk
- User : pwuser
- Client IP : 192.168.0.101
- Policy : admin
- Category : news
- Blocked reason : 'Blocked by admin'
- DNS query type : 33

- Go index -
Performance tuning guide
Although NxFilter is designed to handle several thousands users easily there are several factors you can adjust to get the best performance from NxFilter.

Memory size

At default NxFilter uses up to 512 MB RAM. This is enough for most users. But if you allocate a bigger memory to NxFilter you can expect a better performance. In NxFilter startup scripts, '/nxfilter/bin/startup.bat' you have the following line.

    java -Djava.net.preferIPv4Stack=true -Xmx512m -cp "%NX_HOME%"\nxd.jar;"%NX_HOME%"\lib\*; nxd.Main

If you want to increase it to 1 GB then change '-Xmx512m' to '-Xmx1024m'.

* When you run NxFilter as a Windows service you need to reinstall the service with the increased memory size. When you reinstall it, uninstall the service with 'unstsvc.bat' first and edit 'instsvc.bat' with the increased memory size and then reinstall it by running 'instsvc.bat'.

Disk space and reducing the amount of log data

NxFilter has various reporting features. You can view all the logging data and daily, weekly report and per-user report. However this kind of reporting consumes a lot of disk space. When you have a bigger size of reporting data your system might experience a performance issue.

If you have more than several hundreds users it might be better to have at least 10 GB of disk space for traffic DB. Or to save the disk space you can reduce the amount of data. To reduce the amount of traffic data you can adjust the value for 'Log Retention Days' on 'Config > Setup'.

The other way of reducing the amount of traffic data is to make a whitelist with 'Bypass Logging' option for the domains you are not interested in.

Use client cache for DNS response

NxFilter manipulates DNS cache TTL to clear out the client DNS as soon as possible to make your policy update being applied faster. But basically this is not a critically needed function and it increases the number of DNS queries from your client systems. For the performance, you would better not to manipulate DNS cache TTL.

You can set 'Max Client Cache TTL' in 'Config > Setup' to '0' to turn off the function. When you turn it off your client system will not make a DNS query against the domain already existing in its cache and it will reduce the load for NxFilter significantly. When you have more than 1,000 users we recommend you to turn off this function.

Increase the number of request handlers

NxFilter is a multi-threaded program. It has worker threads handling client DNS requests. The default number of request handlers is 8 and it is enough for most cases. But if you think your NxFilter responding slowly you can try to increase it. To increase it to 16 add the following line into '/nxfilter/conf/cfg.properties' and restart NxFilter.

    rh_num = 16

Using local recursive DNS server

One of the possible cause of performance degrading for NxFilter would be the latency to its upstream server. This is not the case when you have just several hundreds users as NxFilter has its own caching. But if you have several thousands users this could be an issue. So we added local recursive DNS option.

However this doesn't mean that NxFilter does recursive DNS query by itself. Rather you install a recursive DNS server into the server having NxFilter already installed and make NxFilter uses it as its upstream DNS server. If you install something like MaraDNS's Deadwood recursive DNS server and set it to use 10053 port and listening '127.0.0.1' then you need to add the following line into '/nxfilter/conf/cfg.properties' file.

    local_resolver_port = 10053

And then restart NxFilter.

Disable data sharing between cluster nodes

When you have a cluster, there is a massive amount of communication between nodes for data sharing. This could be a performance degrading factor when you have a busy server. To reduce the amount of communication read Clustering with NxFilter section on this tutorial.

- Go index -
FAQ
These are frequently asked questions about NxFilter.

I can bypass NxFilter by accessing websites using IP address.

There are people saying that DNS filtering is useless as they can access a website using IP address. This is a very naive thought and simply not true. In today's Internet environment most websites are running on a virtual host. This means there are multiple websites on one IP address. You can't access these websites without domain.

And the other thing you need to think about is that there are many URLs in a webpage. This is especially true when it comes to a big portal site. Those URLs are based on DNS as well. It is like that you can try to access a blocked website using an IP address but what you get is just a brocken webpage.

* NxFilter can block IP host in URL with its local proxy agents.

- Go index -

It doesn't get blocked/unblocked right away.

This is most likely from the DNS cache on your system. If you are on a Windows system there are two kinds of DNS caches. One is from your browser and the other is from your Windows OS. Before the cache expires your policy change for blocking/unblocking will not be working. Both caches expire eventually but you might want to clear it out immediately. If it is a browser cache you can clear it out by restarting your browser.

If you want to clear out your Windows DNS cache, use the following command on CMD.

ipconfig /flushdns

Normally DNS cache from Windows expires in a day at the maximum. Of course it depends on TTL from DNS record but I have not seen it bigger than 86,400 seconds(1 day) usually. About the browser cache it may take several minutes to get expired. But it will get expired and blocked eventually. So in practice this is not a problem as you don't need to block/unblock a site many times a day.

- Go index -

How do I force users to be filtered by NxFilter?

If you have a firewall in your network it is simple. You just need to block outgoing UDP/53, TCP/53 traffic except from NxFilter. And then you use DHCP to set up NxFilter to be the DNS server for your network. Now NxFilter became the only DNS server that your users can use and their DNS setup to point NxFilter will be done automatically.

- Go index -

How NxFilter determin which policy to apply for a user?

You can assign a policy to a user directly. If the user belongs to a group then the group policy overrides the user policy. This is simple so far. But when you import users from Active Directory there might be users belonging to multiple groups. You don't know which policy to apply to a user in this case.

To solve this problem we introduced 'Priority Points' concept. If there are multiple groups and if they have several different policies, the policy having the highest priority points will be applied. You can set this priority points on a policy. When you want to find out which policy being applied to a user use 'TEST' button on 'User & Group > User'.

- Go index -

What is the quickest way of blocking 'facebook.com'?

Add '*.facebook.com' into 'Whitelist > Domain' and check 'Admin Block' option.

- Go index -

I want to block 'facebook.com' only for students.

You need to be able to differentiate your students on NxFilter with authentication first. And then block 'Social Networking' category on a policy when you use Jahaslist. Then assign the policy to the user or group for your students.

- Go index -

I want to allow sales department to use the Internet freely at lunchtime.

Create a user or a group for your sale department and define a free-time in 'Policy & Rule > Free Time' then assign a free-time policy which is more lenient to the user or group.

- Go index -

How do I change NxFilter's webserver port?

You can change HTTP/HTTPS listening ports on NxFilter. However when you change HTTP port you will lose your block-page redirection. It is because when NxFilter redirects a user there needs to be something waiting for his/her browser on TCP/80 port.

To change the ports, you need to modify these two parameters on '/nxfilter/conf/cfg.properties' file.

http_port = 80
https_port = 443

After you change the ports, restart NxFilter.

- Go index -

How do I reset admin password?

We have '/nxfilter/bin/reset_pw.sh' script to reset admin password. Once you run the script, the admin name and password will be reset to 'admin'. You need to run the script while NxFilter working.

* There is '/nxfilter/bin/reset_acl.sh' to reset access restriction to GUI as well.

- Go index -

Can I bind NxFilter to a specific IP address?

You might want to bind NxFilter to a specific IP address to avoid of having a port collision problem. You can bind NxFilter to a specific IP address using 'listen_ip' parameter in '/nxfilter/conf/cfg.properties' file. If you set it to '0.0.0.0' NxFilter will listen on all the IP addresses of the system but if you set it to a specific IP address NxFilter will listen on the specified IP address only.

* Even if you bind NxFilter to a specific IP address you can not run multiple NxFilter on the same machine. This is because NxFilter needs to bind several ports on localhost for internal communication.

- Go index -

How do I bypass my local domain?

On 'DNS > Setup' You can set your local DNS server and local domain. With this setup if there are DNS queries for your local domain NxFilter forwards the queries to the local DNS server and bypass authentication, filtering and logging.

- Go index -

Can I use an exact matching keyword for log search?

You can use square brackets for exact matching on log search.

    ex) [john], [192.168.0.1]

- Go index -

Why do I need to re-login after lunch break?

Your login session has been expired. If there is no activity(DNS query) from your PC for a certain time your login session expires. You can increase 'Login Session TTL' on 'Config > Setup'.

* If you use single sign-on with Active Directory you can avoid of having this problem.

- Go index -

How do I apply my own SSL certificate?

We use an embedded Tomcat 7.x as the built-in webserver for NxFilter. If you want to apply your own SSL certificate with Tomcat there are two parameters you need to set in Tomcat config file. One is 'keystoreFile' and the other one is 'keystorePass'. However we don't have a separated config file for Tomcat. We use '/nxfilter/conf/cfg.properties' file to set these parameters.

keystore_file = conf/myown.keystore
keystore_pass = 123456

* About how to build keystore file read Tomcat manual.

- Go index -

How do I enable debug mode?

When there is something wrong with NxFilter the first thing you can do is to find out what is going on exactly with its log data. NxFilter keeps its system log data inside '/nxfilter/log' directory. If you need more detailed log data, enable debug mode on '/nxfilter/conf/log4j.properties'. Change 'INFO' to 'DEBUG' inside the file and restart NxFilter.

- Go index -

How do I hide SSL warning?

When a browser being redirected on HTTPS it warns users that they are being redirected. This is for preventing 'Man in the Middle' attack. That is why you get an SSL warning page instead of NxFilter block-page. Your browser is just doing its job and we don't want to interfere that. However we know that there are users wanting to hide the warning page for some reason. While we still can't show the block-page on HTTPS but you can hide it by changing HTTPS port of NxFilter. If you use a non-standard HTTPS port, your users will only see 'Connection Error' message.

To change HTTPS port for NxFilter modify the following line on '/nxfilter/conf/cfg.properties' file.

https_port = 443

- Go index -

I don't see any username on 'Logging > Request'.

The first thing you need to check would be 'Enable Authentication' option on 'Config > Setup'. Some people don't understand that they need to enable authentication before implementing any authentication method.

- Go index -

How do I bypass logging completely?

For internal purposes, the minimum log retention period you can set is 3 days. But you can bypass logging completely by setting 'syslog_only' option on '/nxfilter/conf/cfg.properties' file. If you set this option without having Syslog exportation setup then NxFilter bypasses logging and not sending Syslog data as it doesn't know where to send it.

To enable 'syslog_only' option add the following line on '/nxfilter/conf/cfg.properties' file,

syslog_only = 1

* You still get the counting data but the actual logging data will not be stored into your traffic DB.

- Go index -

How to set up a time zone.

Some of our users reported that they have a different time zone on NxFilter from the system. This happens mostly on CentOS. When you need to set up a time zone for NxFilter manually. You can do that on JVM level. On '/nxfilter/bin/startup.sh' set the following parameter.

-Duser.timezone=Europe/Rome

- Go index -

My Browsers keep restarting after NxClient starting.

NxClient is a local proxy so it needs to update the system proxy settings to redirect HTTP/HTTPS traffic of your browsers to itself. And after it updates the proxy settings it needs to restart the browsers to apply the changes. But you might have another Windows program preventing the update or doing the update for itself. You have a race condition here. To fix it, you have to disable one of them.

- Go index -