NxFilter Tutorial
1. Getting started

2. Blacklist and domain categorization

3. Authentication

4. GUI overview

6. Working with agent

7. NxFilter-Cloud

8. Misc

9. FAQ
System requirements
- Windows, Linux, FreeBSD or other OS having Java(JRE) 1.6 or higher installed.
- 512 mega bytes of system memory.
- 4 giga bytes of disk space.
- 53 port on UDP, 80 and 443 port on TCP.

* You can run NxFilter on even lesser hardware when you have small number of users but we recommend you to have more than 1GB of memory and 40GB of disk space when you have more than 1,000 users. At default NxFilter uses up to 512MB memory. You can increase the limit of memory allocation to NxFilter in its startup scripts. In '/nxfilter/bin/startup.bat' or '/nxfilter/bin/startup.sh' you can modify the value of '-Xmx512m'.

- Go index -
Install NxFilter on Windows
NxFilter provides a Windows installer. When you download and run 'nxfilter-x.x.x.exe' you will view the following display.

After you follow several steps on the installer, it will try to create NxFilter service. If you see the following message you have NxFilter successfully installed.

To access admin GUI, start your browser and type 'http://localhost/admin' into the address bar. Or if you created a shortcut for web-admin in the installation process you can click it. If you see the following login screen your NxFilter is up and running. The initial admin name and password is 'admin' and 'admin'.

The next thing you need to do would be updating the blacklist of NxFilter. You can follow the instruction from What is a blacklist?

- Go index -
Install NxFilter on Linux
When you install NxFilter on Linux system.
- You need to have root privilege.
- Make sure that your system has Java 1.6 or higher installed.
- You need to make the script files excutable using 'chmod +x /nxfilter/bin/*.sh'.
- You can start NxFilter as a daemon use '-d' option for 'startup.sh'.

1. Download 'nxfilter-x.x.x.zip' file from www.nxfilter.org.

2. Extract the zip file into '/nxfilter'.

3. Go to '/nxfilter/bin' and make script files excutable using 'chmod +x *.sh'.

4. Run 'startup.sh'.

5. To access admin GUI, start your browser and type 'http://your-nxfilter-ip/admin' into the address bar. If you install it on '192.168.0.100' type 'http://192.168.0.100/admin'. The initial admin name and password is 'admin' and 'admin'.

* You might want to start NxFilter automatically when your system startup. On our Ubuntu system we have '/nxfilter/bin/startup.sh -d' in '/etc/rc.local' script. You need to use '-d' option for running NxFilter as a daemon.

- Go index -
Install NxFilter on Windows manually
This is about how to install NxFilter on Windows manually using 'zip' package. You also can make it as a Windows service using a batch script included in the package.

1. Download 'nxfilter-x.x.x.zip' file from www.nxfilter.org.

2. Extract the zip file into 'c:/nxfilter'.

3. Go to 'c:/nxfilter/bin''.

4. Run 'startup.bat'.

* If you want to install NxFilter as a service run 'c:/nxfilter/bin/instsvc.bat'. It will create NxFilter service. When you unstall it run 'c:/nxfilter/bin/unstsvc.bat'.

* To run NxFilter as a service 'net start NxFilter'. To stop it 'net stop NxFilter'.

* Use 'net start NxCloud' and 'net stop NxCloud' for NxFilter-Cloud.

- Go index -
Start and stop NxFilter
There are several utility scripts for NxFilter in '/nxfilter/bin' directory.

- To start NxFilter : startup.sh
- To stop NxFilter : shutdown.sh
- To see if it is running : ping.sh

On Windows you can use '.bat' files instead of '.sh' files.

* If you installed it on Windows using Windows Installer you probably have it as a service. To start and stop NxFilter service use 'net start NxFilter' and 'net stop NxFilter'.

* Use 'net start NxCloud' and 'net stop NxCloud' for NxFilter-Cloud.

- Go index -
Client DNS setup
After you install NxFilter you want to monitor and filter Internet activity in your network. To monitor/filter Internet activity you need to make NxFilter to be the DNS server for your network. Since NxFilter is basically a forwarding DNS server you can use the same way of setting up a DNS server for user systems.

The simplest way of setting a DNS server for your users would be modifying the network setup on OS level like the above. But you don't want to setup all the PC in your network one by one. So the best way would be using DHCP server. You just need to modify DNS server address on your DHCP server setup and then your users will be using NxFilter as their DNS server after they get their IP addresses from the DHCP server.

If you have a firewall you can force users to use NxFilter as their DNS server by blocking outgoing traffic on 53 UDP/TCP port. Now NxFilter became the only DNS server your users can use.

- Go index -
Deploying NxFilter in Active Directory
When you deploy NxFilter in Active Directory environment there are several things you need to consider. The first thing is how to implement single sign-on with Active Directory. It's explained on the authentication part of this tutorial. And the other thing is that you might have some trouble for resolving hostnames on Active Directory. This is because in Active Directory the Microsoft DNS server on domain controller is playing some important role and when you just replace your existing DNS server with NxFilter your client PC might have some problem with Active Directory integration as NxFilter doesn't know the hostnames in your Active directory.

To address this issue NxFilter tries to resolve your AD domain against MS DNS server automatically. If you import your Active Directory user and groups on 'User > active-directory' NxFilter will resolve your local domain against AD DNS server without any extra setup. But if you have more than one local domain or deploying NxFilter in Active Directory without importing users then you need to setup zone-transfer or bypass your AD domain from NxFilter.

* Before we came up with the auto-resolving for AD domain we were using zone-transfer for resolving AD domain hosts. When you set up your local domain to be transfered into NxFilter on 'Config > zone-tranfser' it imports your domain using 'IXFR' protocol and resolves your AD hosts on its own.

* If you setup zone-transfer for Active Directory domain you need to allow zone-transfer on your MS DNS server side. If your domain is 'rainbowx.local' then you need to allow zone-transfer for these 2 domains.

rainbowx.local
_msdcs.rainbowx.local

The other way of solving this problem is to use 'local_dns' and 'local_domain' parameters on '/nxfilter/conf/cfg.properties' file. For 'local_dns' the IP of your local DNS server and for 'local_domain' your local domains which you want to bypass from NxFilter. You can have multiple domains separated by commas.

local_dns = 192.168.0.101
local_domain = rainbowx.local, mydomain.local, yourdomain.com

- Go index -
When NxFilter not starting
When you find your NxFilter not starting, the first thing you need to do is checking '/nxfilter/log/nxd.log' file You can find some information about the cause of the problem. The other things you need to check out would be the port collision and Java installation. NxFilter uses 80, 443 on TCP and 53 on UDP. This means NxFilter itself is a DNS server and a webserver. So if you have another DNS server or webserver running on the same system NxFilter will not start.

About the Java installation, if you use the Windows installer in most cases you will not have the problem but if you install NxFilter manually or if you start it manually not using Windows service you might have some Java related problems. To avoid of having this kind of problem there should be Java installed on the system and you need to have proper environment variables for Java.

If your are on Windows system. You will see this kind of message on the command prompt when you type 'java' if you have properly configured Java.

On Windows system you can set these environment variables.

JAVA_HOME = C:\Program Files\Java\jdk1.6.0_38
PATH = %JAVA_HOME%\bin;C:\bin

If it's on Linux NxFilter firstly looks for java in '/usr/bin' and then '/usr/local/bin' so if you don't have java in these directories you need to modify the script files in '/nxfilter/bin' directory or you need to include the path into the environment variables for your system.

To setup 'PATH' system variable for Java you can follow the instruction from the link below.

    - http://java.com/en/download/help/path.xml

- Go index -
What is a blacklist?
A blacklist is a database of categorized domains. This is an essential part of a webfilter for blocking websites by categories. NxFilter supports 3 blacklists.

1. Shallalist
Free for non-commercial use. It has 74 categories. It is maintained on www.shallalist.de. Auto-update tool supported.

2. URLBlacklist
This is a commercial blacklist having 74 categories. It's slightly bigger than Shallalist. You can download it from www.urlblacklist.com

3. Zvelo
The best blacklist in the market. It has well organized 53 categories. Many commercial webfilters are using Zvelo DB. Probably the biggest DB in the market as well. One of the good things is that NxFilter uses its cloud option so it doesn't require import or update.

- Go index -
Updating Shallalist
Updating Shallalist is very easy since NxFilter provides an auto-download and update script. To update Shallalist stop NxFilter and run '/nxfilter/bin/update_sh.bat'. NxFilter firstly checks if there's any update from www.shallalist.de and tries to download the blacklist file and then load it into NxFilter DB. Depending on Internet speed It may take several minutes to finish the whole process.

If you need to update it manually download http://www.shallalist.de/Downloads/shallalist.tar.gz and extract it into '/nxfilter/shallalist1/BL' then run 'update_sh.bat /nxfilter/shallalist1/BL' command.

cd /nxfilter/bin
update_sh.sh /nxfilter/shallalist1/BL

- Go index -
Updating URLBlacklist
This is a commercial blacklist. You need to download it and load it into NxFilter DB by yourself.

1. Download the file from http://www.urlblacklist.com/?sec=download
- What you need is 'bigblacklist.tar.gz' file.

2. Extract the file into '/nxfilter/blacklists'.

3. Stop NxFilter.

4. Run '/nxfilter/bin/update_bl.sh'.

5. Start NxFilter. Now you can view 74 categories in 'Policy > policy > edit'.

- Go index -
Using Zvelo cloud DB
To be compared to those expensive commercial webfilters the only weakness of NxFilter is its relatively smaller blacklist DB. When we use Shallalist, even though it has several millions of domains categorized it is just not enough to cover today's huge Internet. As a result we see a lot of 'unclassified' domains on NxFilter. This means our filtering by category can't be the best.

We always knew that Zvelo DB, http://www.zvelo.com is the best in the market and we wanted to provide it as an option to our users. But we couldn't cover the cost. However one of our OEM partners which is SaferNet, http://www.safernet.co.za based in South Africa agreed to provide some amount of their Zvelo license with very reasonable pricing. So now we can provide Zvelo option to our users and you will find NxFilter using Zvelo DB as the best of all the webfilters. It requires no update or no import and it covers most of the domains existing. Almost no unclassified domains on your log view!

Currently the price for Zvelo license is 3 USD per-user, per-year. If we consider the most webfilter companies having their pricing over 15 USD per-user, per-year this is far beyond being reasonable. If you are interested in buying Zvelo license contact us using 'support @ nxfilter.org' we will give you our OEM partner's Paypal account. Or click the link below.

* After you bought the license contact us using 'support @ nxfilter.org' to get your license file.

The minimum user number for a license is 20 user. So if you have 20 user, the subscription fee will be 60 USD per-year. If you want to see its quality of categorization visit our demo site. We now run our demo site using Zvelo DB. If you want to try it before buying it we can arrange 14 days trial license for you.

Applying your license

When you receive a license file which is 'license.lic' copy the file into '/nxfilter/conf' directory and restart NxFilter. You will see your license activated on 'Category > system'.

Counting user number

NxFilter counts the number of logged-in users or client IP addresses daily basis. If either of them exceeds the specified user number on your license any unlicensed users will be appeared as blocked on your log view. And those domains queried by the unlicensed users will be categorized as 'Unclassified'. However since it is a warning measure this blocking is not actually happening on user side.

* To find out the number of users in your network use daily report on 'Report > daily' or the usage report for last 30 days on 'Report > usage'.

- Go index -
NxFilter and authentication
NxFilter provides several authentication methods including Active Directory integration or single sign-on.

Why authentication

When you install NxFilter first time you only have one policy and it applies to all the users in your network. But what if you are working for a school as a sys-admin and you want to apply different policies to students and teachers. For students more strict policy and for teachers bit lenient policy. Now you need to differentiate users. That's when you need to enable authentication.

Which authentication

NxFilter supports several ways of authentication. You can choose one of them or mix and match some of them.

1. IP based authentication
This is the simplest form of authentication. When you use static IP address for your client PC this might be the best choice. Just associate the IP address of the client PC to the user you create on NxFilter GUI. You also can create a user covers a certain IP range.

* Many people try to use IP based authentication without enabling authentication on 'Config > config'. But IP based authentication is still a method of authentication so you must enable it first.

2. Password based authentication
When you enable authentication NxFilter blocks user request with its login-page unless the user already logged in or having IP address associated to him/her. To go through the login-page your users need to enter his/her username and password. You can set this password to a user on NxFilter GUI after you create a user. Means you can have password based authentication without Active Directory or LDAP.

3. LDAP based authentication
If you have OpenLDAP or Active Directory your users can go through the login-page using their LDAP credentials. To use this feature you need to import your users from your LDAP server first.

4. Login-token based authentication
NxFilter has a special concept called 'login-token'. This is used for remote user authentication or filtering. This login-token being created for each user when you create or import users. You can use this login-token for NxClient when you need remote filtering or dynamic IP update.

5. Single sign-on against Active Directory
Many people want to deploy their webfilter transparently. Or you don't want to show any login prompt to your users. So NxFilter provides Active Directory integration. Once you set it up your users don't need to go through login-page and your users will appear on NxFilter GUI with their AD username and group.

- Go index -
Single sign-on with Active Directory using NxLogon
When you have Active Directory you want to have single sign-on against Active Directory and not showing any extra login prompt to users. For this we have a client program that is NxLogon. When you run NxLogon on a user PC it creates and keep a user login session on NxFilter.

However you don't want to install and manage this program on every PC in your network. So we use the logon script of GPO(Group Policy Object) on Active Directory. This logon script is being executed whenever a user logon to Active Directory domain. And we can write our own logon script to launch NxLogon and then we get the single sign-on.

* If you want to have single sign-on against Active Directory you firstly need to import users and groups from your Active Directory. To import users and groups read, GUI - User

After you launch NxLogon it will create a login session for the Windows user account and it will refresh the session on every minutes. You can follow these steps to launch NxLogon from GPO.

1. Download nxlogon-4.x.zip package from www.nxfilter.org.

2. Modify IP address in 'nxlogon.bat' to point your NxFilter IP address. If you use clustering use your master node IP.

3. Open 'Administrative Tools > Active Directory Users and Computers' on your DC.

4. Open 'Group Policy' tab in properties of your Active Directory Domain.

    

5. Click 'Edit' button and then go to 'User configuration > Windows Settings > Scripts (Logon/Logoff)'.

    

6. Click 'Logon' and click 'Add' and then click 'Browse' button. You will see 'Logon' directory to select file. Copy your 'nxlogon.bat' and 'nxlogon.exe' from NxLogon package into 'Logon' directory. You can drag and drop the files into the directory.

7. Select 'nxlogon.bat' which you copied into 'Logon' directory as the logon script to add.

    

8. Now every time users logon to their systems 'logon.bat' will be executed and it will launch 'nxlogon.exe'. You can check the process running on Windows task manager.

    

* Since NxLogon is running background you can't see it running. If you want to check if it's running see 'Processes' tab on 'Windows Task Manager'.

* If you want to remove the login session immediately after user logout use 'nxlogoff.bat' as a logoff script in GPO.

* Rob Asher wrote a single sign-on script from Linux and Mac clients using NxFilter's login API for custom login script.
     NxFilter + OS X and linux login script

- Go index -
Single sign-on with Active Directory using NxMapper
While using NxLogon is still the best solution for AD single sign-on but some people find it difficult to setup all these GPO and logon script for launching NxLogon. So we offer an easier way of implementing single sign-on against Active Directory. When you install NxMapper on your domain controller it will grab username and IP address pair and send it to NxFilter.

* If you want to have single sign-on against Active Directory you firstly need to import users and groups from your Active Directory. To import users and groups read, GUI - User

Install and run NxMapper

We offer a Windows installer for NxMapper. It will install NxMapper as a Windows service. After you install NxMapper using the installer you will have its setup program running.

* You can add multiple IP addresses separated by commas if you run a cluster of NxFilter.

After you modify the config value test your setup first and then start it.

Differences from using NxLogon

Although it's a lot easier compared to using NxLogon, NxMapper also has its own limits. Firstly you can't use the application control function from NxLogon.

The other thing is that the login session can be expired. While NxLogon refreshes the login session on evey minutes, NxMapper creates or refreshes user login session only when there's user activity on DC. So if your users don't use Internet for a while then their session will be expired. Once the session expired your users will be redirected to the login-page of NxFilter.

To prevent the expiration of the login session you can increase the session timeout value on 'Config > config > Block and authentication > Login session TTL' on the GUI of NxFilter. If you increase it to 120 minutes it will be enough to cover the lunch break. After your users resume using the Internet then the session continues.

Terminal server exclusion

When we use NxMapper we might have some problem with terminal server. If there are multiple users on one system the IP address of the system will be associated to the user whose action detected lastly by NxMapper. It means your users can be appeared on NxFilter with a different username. To prevent having this kind of problem the best solution would be creating an IP based user for your terminal server.

* NxMapper needs to be installed on a domain controller.

- Go index -
Single sign-on with Active Directory, OpenLDAP using NxClient
NxClient is basically a remote user filtering client for mobile workers with their own laptop. But you can use it for single sign-on against Active Directory or OpenLDAP. One good thing is that since there's Mac OS version of NxClient you can have single sign-on from Mac OS.

If you already tried to use NxClient you already know that single sign-on using NxClient is possible with its 'login-token' concept. But with this approach the problem is that it's almost impossible to setup several hundreds of NxClient installations with their own 'login-token'.

So we provide a way of running NxClient on local network without setting up different login-token to each client PC. What you need to do is to install NxClient using a common login-token for all the client PC. Then when it starts it will look for its server that is NxFilter on local network and if it finds one it will create a login session for the current logged-in user or console username.

* For NxClient being able to detect local NxFilter, you have to use NxFilter as the DNS server for your client PC.

Another good thing is that since it's running as a Windows service or a daemon on Mac OS X your users can't stop it.

To find out more details about NxClient read this, NxClient and remote user filtering

- Go index -
Custom login script for single sign-on
Currently NxFilter supports single sign-on with Active Directory. However some people need more than that. You might want to have single sign-on from your Linux clients against Active Directory or you might want to have single sign-on with your OpenLDAP users.

NxFilter supports an API set for creating IP session through HTTP protocol or webpage. You need to write your custom login script to call some webpage on NxFilter's built-in webserver. And then your users don't need to see the login-page.

Currently it's on '/nxfilter/example/login_user.jsp'. Initially the access of the page is restricted to localhost only for security reason but you can edit the JSP page to allow calls from your local network.

You can call the webpage this way.

    http://192.168.0.100/example/login_user.jsp?ip=192.168.0.100&uname=john

As you see 2 parameters being passed. One is the IP address of your user and the other one is the associated username. The username should be imported or created on NxFilter side already.

One thing you need to consider when you write your own login script is that it might be better to call the webpage periodically. There is a session timeout concept in NxFilter. If there's no activity from a logged-in user for certain amount of time the login session will be expired. So if you don't want to show your users the login-page you'd need to refresh the login session periodically.

There are 3 methods of UserLoginDao class for custom login script.

- create_ip_session(String ip, String uname) : Creating login session with the IP and username.
- delete_ip_session(String ip) : Deleting login session with the IP.
- find_user(String ip) : You can find a logged-in username by its associated IP address.

All the example JSP pages are in '/nxfilter/webapps/example' directory.

- Go index -
The order of authentication methods
NxFilter uses multiple authentication methods at the same time. If there is a user request NxFilter tries to find its associated user based on the IP address. Firstly NxFilter looks into the memory map for IP based users. If there's no user associated to the given IP address it looks into IP session map which populated by sigle sign-on agent or user login through NxFilter login-page. And if there's still no user found finally NxFilter redirects the user request to its login-page.

This is the order of authentication methods application.

1. Single IP associated user
You can create an IP based user associated with a single IP or an IP range on NxFilter. We put single IP association first so that you can exclude some systems from IP session or single sign-on by creating an IP based user.

2. IP session
When a user login to NxFilter it keeps login session based on user IP and refreshes the session as long as there's user activity. This session can be created by login through login-page or the login agents like NxLogon and NxClient.

3. IP range associated user
Some people want to associate their whole network to a user to apply the same policy. But they still want to apply different policies to their Active Directory logon users. For example you can create a user associated to the IP range of 192.168.0.1 to 192.168.0.255 to cover your entire network. And you apply 'Default' policy to this user. Then you implement single sign-on with Active Directory. Now if there's anybody not having Active Directory account they get applied with 'Default' policy and the other users on Active Directory will be applied with the policies associated to their groups.

- Go index -
GUI - Config
These are mostly system configuration parameters for NxFilter.
Config > config > Block and authentication

- Block redirection IP
Simply speaking, this is the IP address of NxFilter itself. If there is any blocked DNS request, it will be redirected to this IP address. When you first start NxFilter it will attempt to decide its IP address from your system setting.

* You can add multiple block redirection IP addresses separated by commas for load balancing purpose.

- External redirection IP
When you use remote filtering you might need to use a different 'block redirection IP' for remote filtering clients since they are outside of your network. If you leave this blank NxFilter will use 'block redirection IP' for redirecting remote filtering clients.

- Enable authentication
This option is required for AD-integration or any other user authentication method. After you enable this option any unauthenticated users will be redirected to NxFilter login-page. As a result users will be forced to login to use the Internet.

* One thing you need to know is that even if you use only the 'IP based authentication' it is still a method of authentication. So you need to enable this option. Many users are confused with this but since NxFilter uses multiple authentication scheme at the same time we have to keep this policy.

- Login domain
You can access NxFilter login-page using the domain defined here.

- Logout domain
You can clear out user login session using this domain.

- Login session TTL
NxFilter keeps the user login session after a user logged in through its login-page so that your users don't need to see the login-page again while they are using the Internet. But this login session needs to be expired eventually. It is especially required when there is a shared PC by several users. So we have 'TTL' value for this login session. If a user doesn't use the Internet for the specified amount of time here his/her login session expires and the user needs to re-login.

Config > config > DNS setup

- Resolving DNS server
NxFilter is basically a forwarding DNS server. You need to setup the IP addresses of the DNS servers which resolve the DNS queries forwarded by NxFilter. You can have up to 3 resolving DNS servers.

- Resolving DNS query timeout
Timeout for a DNS query to a resolving DNS server.

- Max client cache TTL
Your client PC has its own DNS cache. This client side cache might cause some problem in DNS filtering. Your blocking doesn't work until the cache expires. To reduce the impact from this kind of problem NxFilter provides an option for manipulating client cache TTL. If you set the value to '60' NxFilter modifies the DNS cache TTL to '60' if the TTL is bigger than 60.

0 - Don't touch it.
60 - Don't touch it if it's smaller than 60 and make it '60' if it's bigger than 60.

We introduced this function to minimize the effect from the client cache. However if you have more than 1,000 users you would better turn this function off to have better performance.

- Response cache size
NxFilter has its own cache for DNS query result. Once it has a query result from resolving DNS server it keeps it in the cache until it expires by TTL value from the DNS packet and answering clients with the records from the cache. Generally speaking the bigger cache size would be better for the performance. Currently the default size is 100,000 and it's enough for most sites.

Config > config > Syslog

Like the other enterprise security software NxFilter supports syslog exportation of its log data. You can build your own reporting system with this feature or you can monitor all the logging in real-time way.

- Syslog host
The host address to which you want to send syslog data.

- Export blocked only
With this option NxFilter sends log data of blocked request only.

- Enable remote logging
Enables syslog exportation.

Config > config > Netflow

NxFilter supports bandwidth control. This is possible by importing NetFlow data using the built-in NetFlow collector.
For more detail read this, Bandwidth control with NxFilter

- Router IP
The IP address of the device sending NetFlow data to NxFilter.

- Listen port
The UDP port number of the NetFlow collector.

- Run collector
Run NetFlow collector. After change this option you need to restart NxFilter.

Config > config > Misc

- Admin domain
You can access admin GUI using IP address of NxFilter or 'localhost'. But once you setup admin domain you can access it using the domain you want. For example if you use 'admin.nxfilter.org' as your admin domain you can access your admin GUI by typing 'http://admin.nxfilter.org/admin' into your browser address bar.

* This only works when you use NxFilter as your DNS server. Otherwise you need to register your admin domain to your own DNS server.

- Bypass Microsoft update
You don't want to block Microsoft update with your filtering. Enabling this option means bypassing 'micfosoft.com' and 'windowsupdate.com' and their subdomains.

- Logging retention period
If you keep your log data too long it will eat up your disk space a lot. You can set how long NxFilter keeps its log data here.

- SSL only to admin page
When you want to allow only HTTPS access to admin GUI enable this option. Once you enable this option you will be redirected to the SSL port automatically even if you use HTTP.

- Auto backup
NxFilter makes a backup file for its config into '/nxfilter/backup' directory on '01:00' everyday. The name of the backup file starts with 'auto-' prefix. You cat have up to 30 backups.

- Agent policy update period
NxFilter supports application control and proxy filtering through NxClient. NxClient fetches the policy for application blocking and proxy filtering according to the period defined here.

Config > admin

You can change admin name and password for GUI login here.

Config > config > Alert

NxFilter sends alert email for recent blocking or clustering node down incidents. For example if you want to send an alert email to 'support @ nxfilter.org' from 'nxfilter200 @ gmail.com' on every 15 minutes then the setup would look like below.

- Admin email : support @ nxfilter.org
- SMTP host : smtp:gmail.com
- SMTP host : 465
- SMTP SSL : on
- SMTP user : nxfiter200
- Alert period : Every 15 minutes

Config > allowed IP

NxFilter has IP based access restriction function for its DNS, GUI, login redirection. You might want to use this feature when you put your NxFilter on a public IP address. You can make whitelist/blckalist way of ACL here.

Config > backup

You can make a backup for config DB of NxFilter manually. The backup files will be created into '/nxfilter/backup' directory.

Config > redirection

Domain to IP or domain to domain redirection is possible with NxFilter. It works like custom DNS record.

Config > zone-transfer

In some situation you need to import DNS zones from the other DNS server. Once you setup zone-transfer here NxFilter imports the DNS zone on every minutes using IXFR protocol.

Config > block-page

This is the setup for custom block-page, login-page, welcome-page. When you edit your block-page you can use the following variables populated by NxFilter for making your block-page more informative.

- #{domain} : Blocked domain
- #{reason} : Reason for block
- #{user} : Logged-in username
- #{group} : Groups of the logged-in user
- #{policy} : The applied policy
- #{category} : Categories or the blocked domain

Config > cluster

NxFilter has a built-in clustering. You can make your NxFilter to be a master node or a slave node in a cluster. After you change the values in cluster setup you need to restart your NxFilter to apply new settings.

- Go index -
GUI - User
You can create or import users and groups here. NxFilter supports user importation from Active Directory and OpenLDAP.

Creating a user

You can create a user on 'User > user'. Once you create a user you can edit the user properties on the edit view. There are 3 types of users in NxFilter

1. IP user
It has associated IP addresses or IP ranges and will be authenticated based on IP address.

2. Password user
If you set password for a user it becomes a password-user. You can use the password on the login-page of NxFilter.

3. LDAP user
When you import users from your LDAP servers or Active Directory they become LDAP users. They can use LDAP or Active Directory user credentials on NxFilter login-page.

Properties of a user

- Password : The password for login through NxFilter login-page.
- Work-time policy : The policy being applied during the work-time.
- Free-time policy : The policy being applied during the free-time. You can define a free-time on 'Policy > free-time'.
- Expiration date : The expiration date for a user account.
- Login token : The access token for NxClient. It is created when a user created or imported.

Testing a user

When you have LDAP imported users you may have multiple groups and policies for a user. As a result it becomes difficult to figure out which policy a user fall into. This is especially true when you apply different policies for free-time. To find out which is the 'applied policy' for a user use 'test' button on user list. It fetches the information of user state from NxFilter real time way so you get the correct information about how a user being handled by NxFilter.

* You can use this test view to find out how much quota or bandwidth consumed by a user or to reset quota or bandwidth for a user.

Creating a group

You can create a group on 'User > group'. After you create a group you can setup a policy for the group by editing the group. You also can assign members to a group on edit view.

* As of v2.0.5 you can define group specific free-time on 'User > group > edit'. When a user belongs to multiple groups and one of the groups fall into free-time range NxFilter applies the free-time policy for the user. NxFilter applies group specific free-time first and then global free-time next.

Importing users and groups from Active Directory or OpenLDAP

You can import users and groups from Active Directory on 'User > active-directory'. For example if you have Active Directory with the following setup.

- Domain controller : 192.168.0.100
- Domain : nxfilter.local
- Admin username : Administrator

Then you can create Active Directory import setup with the following details.

- Host : 192.168.0.100
- Admin : Administrator@nxfilter.local
- Password : your-password
- Base DN : cn=users,dc=nxfilter,dc=local
- Domain : nxfilter.local

After setup your AD detail you can import users and groups by using 'import' button immediately. You also can setup a periodical import by selecting auto-sync interval.

* When you import users from Active Directory you may have a users belongs to multiple groups and having multiple policies as a result. If you want to select one policy over to the others use 'priority points' property of a policy. Bigger priority points wins over the other policies.

* If you want to find out if there is any issue with the connectivity between NxFilter and your DC, use 'test' button.

* NxFilter also supports LDAP importation from OpenLDAP.

- Go index -
GUI - Policy
With NxFilter you can have multiple policies based on user and group.

Creating a policy

When you install NxFilter there is only one policy that is 'Default'. This policy will be applied to all the users if you don't make any change on NxFilter setup. If you want to apply different policies to different users you need to create other policies and enable authentication.

Editing a policy

After you create a policy you can modify its properties.

- Priority points
If there are multiple policies associated to one user then the policy having the highest points will be applied.

- Enable filter
If you disable this option there will be no blocking from the policy.

- Block all
Block all. Global whitelist overrides this one still.

- Block unclassified
Block uncategorized domains.

- Ad-remove
Block domains in 'adv' category of Shallalist with a blank block-page.

* This is useful when you want to remove embedded adverts without showing NxFilter block-page.

- Max domain length
There are some malwares using domain name itself as a communication tool or a message protocol. These domains are abnormally long while the length of most domains are under 30 characters. You can set a limit for the length of a domain for blocking these abnormal domains. To prevent having false positives NxFilter doesn't apply 'Max domain length' against 30,000 well known domains.

- Block covert channel
Some malwares or botnets are using DNS protocol as their communication tool. They are using DNS query and response to communicate to each other.

- Block mailer worm
Normally you are not supposed to see MX query from your client PC. When NxFilter finds MX type query from your client PC it will be regarded from some malware trying to send emails.

- Block DNS rebinding
When NxFilter finds a private IP address(192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8) on DNS response packet it will be blocked as DNS rebinding attack.

* If you have your own DNS record with private IP address you need to bypass the domain on whitelist.

- Allow 'A' record only
This is the most strict way of filtering malwares and botnets employing DNS protocol as their communication tool. If you are an ordinary office worker you don't need to use any special types of DNS records. With this option NxFilter allows A, AAAA, PTR, CNAME and the other types of DNS records will be blocked. If you need to allow the other types of records from some of your users then you need to apply a different policy to them.

- Quota
NxFilter has quota-time feature. You can allow your users to browse specific sites for certain amount of time. You can set the amount of time to here.

- Quota all
Apply quota to all domains including unclassified domains.

- Blocked categories
You can block user request by categories. These categories are imported from Shallalist or custom categories.

- Quotaed categories
If you check certain categories in 'Quotaed Categories' then your users can access the websites in the categories for the amount of time you specified with 'Quota' above. When a user consumed up his quota his/her DNS requests for those sites will be blocked.

- Safe-search
Enforcing safe-search against Google, Bing, Yahoo search engines and Youtube.

* Yahoo and Youtube safe-search requires local proxy agent filtering.

- Disable application control
Disable application control on policy level.

- Disable proxy filtering
Disable proxy filtering for on policy level.

- Logging only
Monitoring user activity without blocking them.

Define a free-time

You can define a global free-time in 'Policy > free-time'. If you assign a free-time policy to users it will be applied during the time defined here. You can have multiple free-time.

* If the start-time is bigger than the end-time then it will break into 'end-time ~ 24:00' and '00:00 ~ start-time' on the same day.

Application control

NxFilter provides application control through its agents, NxLogon and NxClient. For more details read 'Application control' section of this tutorial.

Proxy filtering

NxFilter provides proxy filtering through NxClient. For more details read 'Proxy filtering' section of this tutorial.

- Go index -
GUI - Category
On NxFilter there are system categories and custom categories. System categories are the domain categories defined by your blacklist DB or domain categorization DB. And custom categories are the categories you can create. Once you create a custom category you can add domains into the category. These categories appear on the policy edit view and you can block domains by these categories.

Currently NxFilter supports several blacklist options for system category. If you want to find out more read 'Blacklist and domain categorization' section of this tutorial.

* If you want to include subdomains into a custom category use asterisk.

    ex) *.nxfilter.org

* If you want to find out which category a domain fall into, use 'Category > domain-test'.

- Go index -
GUI - Whitelist
You can make a global or per-policy whitelist for a domain or keyword here. This one can be used as a blacklist as well by enabling 'admin_block'. If there is a domain having 'bypass_filter' and 'admin_block' options enabled together then 'admin_block' overrides 'bypass_filter'.

- Bypass authentication : When you enable authentication some of your application may not be working. It is because some application needs to access Internet without user attendance. In that case you can try to bypass authentication for the related domains.

- Bypass filtering : When you want to exclude some domains from your filtering policies use this option.

- Bypass logging : Sometimes you find that there are too many log data from some domain which you are not interested in. Or you want to save your disk space by excluding some domains generating too many log data.

- Admin block : This works as a blacklist. When you want to block some domains regardless of your policy use this option.

* When you use whitelist by domain you can use asterisk to include subdomains.

    ex) *.nxfilter.org

* Since the policy association comes after the authentication 'bypass_authentication' in per-policy whitelist will be ignored.

- Go index -
GUI - Main, Logging, Report
NxFilter can keep its log data up to 90 days and you can generate daily, weekly, per-user reports based on this log data.
Main

When you login to your admin GUI you will see the dashboard of NxFilter. There are several charts for showing summary for the last 2 hours. On the bottom of the dashboard you can see 10 recent block logs for the last 12 hours.

* The difference between request-sum and request-cnt is from NxFilter logging system. To reduce the amount of disk access NxFilter keeps all the log data into its memory space. And then it flushes the data once a minute. If there's the same data it only increases the count for the data. This also helps to prevent DDOS attack attempt to NxFilter when you put it on public network.

Logging

You can search user request log with various conditions in 'Logging > request'. Logging data is being updated once a minute to reduce the load on DB.

In 'Logging > signal' you can view the log of signals from NxClient.

In 'Logging > NetFlow' you can monitor NetFlow data imported.

* Use square brackets for exact matching on log search.

    ex) [nxfilter], [192.168.0.100]

Report

NxFilter generates daily, weekly, per-user report.

- Go index -
Differences between agents
NxFilter provides several agents. Some are for single sign-on against Active Directory. And some are for remote user filtering and dynamic IP update. Some of them can be used for application control and proxy filtering.

1. NxLogon
This is the sigle sign-on agent for Active Directory. You can launch it from the logon script in GPO. It supports application control.

2. NxMapper
Another way of Active Directory integration or single sign-on. Unlike NxLogon you install this one on a domain controller as a Windows service. It detects user logon on Active Directory and creates login session for the user on NxFilter.

3. NxClient
The remote user filtering agent of NxFilter. When you have a mobile worker or home worker you can install NxClient on your mobile worker's laptop. NxClient running as a Windows service and filtering user Internet activity. It supports application control and proxy filtering.

4. NxUpdate
This is the dynamic IP updater for NxFilter.

- Go index -
NxClient and remote user filtering
NxFilter provides a remote user filtering client software that is NxClient. Once you install NxClient onto user system you can filter and monitor the Internet traffic from the user system on your NxFilter GUI centrally regardless of its location. This means you can filter and monitor Internet activity of your mobile worker or home worker.

Installation of NxClient

After you install it using NxClient installer you will have its setup program running. There are 'Server IP' and 'Login token' parameters and you need to replace their values to your own.

* On NxFilter every user has a login-token. You can find it on 'User > user > edit' on NxFilter-GUI.

* NxClient is a Windows service program. It will start at system startup automatically.

* When you install NxClient or NxUpdate on Mac OS X, read Installing NxClient or NxUpdate on Mac OS X on this tutorial.

After you modify the config value test your setup first and then start it. After starting NxClient you can check if it's working by viewing 'Logging > signal' on your NxFilter GUI. There will be signals from your client.

* You can add multiple IP addresses separated by commas if you run a cluster of NxFilter.

* To change the config value run 'C:/Program Files/nxclient/setup.exe'.

Signal of NxClient

When it comes to remote user filtering the most tricky part would be how to force users to be filtered. Nobody wants to get filtered and they are away from the office. If they use their personal PC then you can not filter them anyway. But as long as they use a company laptop you still can filter them by installing NxClient on the system.

However what if they uninstall or stop NxClient? NxClient is running as a service and it doesn't provide uninstaller for 'Add/Remove programs' in control panel. So if your users don't have enough privilege to modify their systems you don't have these problems.

But sometimes you need to give your users the 'local administrator' privilege. In that case it's not possible to stop users from uninstalling or stopping NxClient. So we defined several signals with which you can find out what's happening on user system. Once you install NxClient on a system it will send these signals to NxFilter and you can view the log of signals on 'Logging > signal'.

- START : When NxClient starts it sends START signal to NxFilter.
- STOP : When NxClient stops it sends STOP signal to NxFilter.
- PING : On every 5 minutes NxClient sends PING signal to NxFilter.

You can view these signals on 'Logging > signal' on NxFilter GUI.

Fail-safe measure for NxClient

You can achieve this by specifying multiple server IP addresses. If one of them fails NxClient talks to the others. However there still can be a time when your NxClient can't connect to all of its server. In that case it bypasses filtering. As soon as the connection restored it will filter again.

Auto-switch between local network and remote network

When you use NxClient on your mobile worker's laptop you might have a problem when they stay in the office with Active Directory integration. Even if you want to apply local network filtering rule based on his AD login username NxClient will be doing its job with its server-ip and login-token.

As a result your mobile worker will be filtered twice. One from NxClient, one from your local NxFilter. And your mobile worker might be required to go through login-page of NxFilter as its login session has not been created.

To address this issue as of v3.4 NxClient supports auto-switch between local network and remote network. This means that NxFilter is intelligent enough to find out if it's on local network or remote network and if it's on local network it will act as a login agent to create its login session on NxFilter and yield to the filtering of local DNS server which is NxFilter.

* Since NxClient is acting as a login agent on local network NxLogon will not start if there's NxClient already working.

* If you don't like auto-switch behavior you can add 'no_switch = 1' line info 'C:/Program Files/nxclient/conf/cfg.properties'.

Uninstalling NxClient

To prevent an accidental uninstall by user NxClient doesn't provide uninstaller for 'Add/Remove programs' in control panel. When you uninstall NxClient you need to do it manually.

- Run 'C:/Program Files/nxclient/bin/unstsvc.bat'.
- Delete 'C:/Program Files/nxclient' folder.

- Go index -
NxUpdate and dynamic IP update
When you have a client system using dynamic IP address and you want to associate its IP address to a specific user you can install NxUpdate on their systems. Once you tie up NxUpdate to a specific user using his/her login-token it will update the his/her associated IP address automatically.

NxUpdate has basically the same structure with NxClient. You can install it in the same way as NxClient.

* It sends START, STOP and IPUPDATE signals.

- Go index -
Application control with NxLogon and NxClient
NxFilter supports application control through its Active Directory single sign-on agent that is NxLogon and its remote user filtering agent, NxClient. You can block unwanted programs by setting up your application control rule on NxFilter GUI centrally and you can find out who tried to run the blocked programs by the log view of NxFilter GUI.

How it works

After you define your application control rule on 'Policy > application' NxLogon, NxClient retrieves the application control policy periodically according to the time defined on 'Config > config > Misc > Agent policy update period'.

* You can adjust the policy update period for NxLogon or NxClient by changing the value for 'Agent policy update period' on 'Config > config'.

Supported options

1. Block by port scanning
NxLogon, NxClient detects UltraSurf and Tor processes by port scanning. This means even if your users change the process name or run them from a USB stick NxFilter can find these processes and block them.

2. Block by process name
NxLogon, NxClient supports 'block by process name'. This works based on keyword matching against process name. You can add your blocked keywords on GUI and If NxFilter finds the matching process name it will block the process.

3. Block by window title
Windows programs are supposed to have a window title. For example Skype has 'Skype' in its window title and uTorrent has 'Torrent' in its window title. You can define your keywords for matching against window title of the blocked programs.

* By default all the keywords are for partial matching but you can specify the exact matching using the square brackets. If you need to add a keyword having spaces then use double quotes.

    ex) Skype [Dropbox.exe] "Tor Browser"

* NxLogon doesn't support Unicode or multi-bytes keywords for application control.

Enable application control only for specific users

Basically the application control of NxFilter works as a global policy. However you can disable the application control for some users by checking 'disable application control' option on the 'Policy > policy > edit' on GUI.

Logging blocked application

NxFilter is basically a dns-filter so its logging structure was designed for showing allowed/blocked domains. To accommodate the log data about blocked application NxFilter introduced these domains or rules.

- ultrasurf.port.app : UltraSurf has been blocked by port scanning.
- tor.port.app : Tor has been blocked by port scanning.
- chrome.exe.pname.app : Chrome has been blocked by its process name.
- Skype.title.app : Skype has been blocked by its window title.

Execution interval

Finding and blocking application may cause some amount of CPU load. If you don't want to cause too much load to your client PC increase 'Execution interval' on 'Policy > application'.

- Go index -
Proxy filtering with NxClient
As of V2.2.2 NxFilter supports safe-search enforcing and URL keyword filtering and the other web-proxy based filtering methods. To enable these features you need to have NxClient running on user PC.

How it works

Firstly you define your proxy filtering rule on 'Policy > proxy' and then after NxClient started on user system they will filter user web traffic by setting up itself as a local proxy server. NxClient retrieves the proxy filtering rule periodically according to the time defined on 'Config > config > Misc > Agent policy update period'

Supported options

1. Block HTTPS
You can block all the HTTPS traffic.

2. Block IP host
Blocking HTTP requests with IP host in URL.

3. Block other browser
NxFilter's proxy filtering is being activated through system proxy settings. Internet Explorer and Chrome are using system proxy already and many other applications are also using system proxy. But there are some applications having their own proxy setup or making direct connection to Internet. With this option enabled NxClient will block any program making direct HTTP connection to the Internet.

* Currently proxy filtering support Internet Explorer, Chrome, Firefox. The proxy setup of these browsers will be updated to use NxClient as their proxy.

* You can allow direct HTTP access to some application using 'Excluded keywords' on 'Policy > application. Although it's for application control it is still effective against 'other browser blocking'. It works based on keyword matching against process name.

4. Blocked keyword in URL
Keyword filtering against URL.

Enable proxy filtering only for specific users

The proxy filtering of NxFilter works globally. If you need to disable it for some users check 'disable proxy filtering' option on the 'Policy > policy > edit' on GUI.

Logging

You will get domain level log data. But you will see a detailed block reason like below.

Domain: www.google.com
Reason: Blocked by proxy, url_kw=game

- Go index -
Installing NxClient or NxUpdate on Mac OS X
When you download and uncompresss the nxclient-x.x-mac.zip file you will find these files in the uncompressed directory.

- install.sh
- nxclient
- org.nxfilter.nxclient.plist

* If you install NxUpdate just change 'nxclient' to 'nxupdate' in this tutorial.

Inside the uncompressed directory, run 'install.sh' with root permission like below,

sudo ./install.sh

'install.sh' will copy 'nxclient' file into '/usr/bin' and 'org.nxfilter.nxclient.plist' into '/Library/LaunchDaemons direcotry'. Now you need to modify the config values inside '/Library/LaunchDaemons direcotry/org.nxfilter.nxclient.plist'.

<string>server = 192.168.0.100</string>
<string>token = GKSYEJYG</string>

For server your NxFilter IP address and for token your user's login token. Then start it.

sudo launchctl load -w /Library/LaunchDaemons/org.nxfilter.nxclient.plist

If you want to stop it,

sudo launchctl unload -w /Library/LaunchDaemons/org.nxfilter.nxclient.plist

When you want to test connectivity or validate config values try this command.

sudo /usr/bin/nxclient "server = 192.168.0.100" "token = GKSYEJYG" -t

When you want to uninstall it,

sudo launchctl unload -w /Library/LaunchDaemons/org.nxfilter.nxclient.plist
sudo rm /usr/bin/nxclient
sudo rm /Library/LaunchDaemons/org.nxfilter.nxclient.plist

- Go index -
What is NxFilter-Cloud?
NxFilter-Cloud is a fully rebrandable multi-tenancy cloud based webfilter software. It is based on NxFilter and inherited the most of the features of NxFilter.

These are the features only available on NxFilter-Cloud.

Multi-level admin

If you want to build your own cloud service one of the essential factors would be being able to create accounts for your customers and the customers need to be able to setup their own policy on their own GUI.

On NxFilter-Cloud there are 3 kinds of users.

    admin > operator > user

Admin is actually the admin of NxFilter-Cloud. It has almost same GUI as in the freeware version but being an admin you can create operator accounts. These operator accounts are for your customers and it's something like a sub-admin on NxFilter-Cloud. They can create and manage their own users and policies.

Creating an operator

To create an operator you need to login to the GUI with admin account. On 'Config > operator' you can create an operator. When you create an operator NxFilter creates a default user and a default policy for the operator with the same name. After you create an operator you can change the number of users and policies the operator can create. This means you can have several levels on your service based on the permission for an operator.

Operator GUI

On NxFilter-Cloud each operator have their own GUI. If you login to the GUI with an operator account you will be on the operator mode GUI. It's a bit restrictive compared to admin GUI as you only can setup operator specific parameters.

Operator and user

Operators can create their own users and apply different policies to each user. Users can be authenticated based on IP addresses or password or using NxClient. If you are on Active Directory environment you can use NxLogon for single sign-on with your Active Directory.

Operator specific dashboard and report

Dashboard and report of NxFilter is still available on operator GUI.

Operator specific free-time

Each operator can define their own free-time and they can setup a work-time policy and a free-time policy for their user.

Operator specific whitelist and blacklist

You can add operator specific whitelist/blacklist based on domain name. But you still have the global whitelist/blacklist for admin. So you can have more flexibility to deal with these whitelist and blacklist as an admin.

Operator specific alert-email

NxFilter-Cloud sends alert email about the blocking incidents to each operator. Operators can setup their email addresses to receive the email and define alert period on 'Config > config'.

* You need to setup global alert email setup first to send the operator specific alert email. You can setup this global alert email on 'Config > alert' of admin GUI.

Operator specific block-page

Each operator can have their own block-page. If there's no block-page defined by operator NxFilter-Cloud shows the default block-page by admin.

Authentication over cloud

NxClient still works against the cloud version. This means you can differentiate users behind their router and you can apply different policies on different users.

Dynamic IP updater

Many of your clients will be using the service from dynamic IP addresses. So one of the essential factors for your cloud based web-filter service would be having a dynamic IP updater. You can use NxClient for this purpose.

Dynamic DNS association

We provide NxUpdate for associating a dynamic IP address to a user. But some of your users may already have a dynamic IP updater for their dynamic DNS service. They don't want to install one more thing on their system. For that reason, we support the association between a user and a dynamic domain. You can add a domain instead of an IP address as an associated IP/domain on user edit page.

Rebranding or customization of GUI

Its GUI layer is designed for easy customization. Its GUI layer is separated from its core part. You just need to modify all the JSP pages in '/nxfilter/webapps' directory. These JSP files have a naming rule corresponding to the GUI menu structure. So it's easy to find which file you need to modify.

- Go index -
Install NxFilter-Cloud
NxFilter-Cloud is designed for multi-tenancy cloud based webfilter service. It has almost every features of NxFilter. You can install and run NxFilter-Cloud in the same way as you do with NxFilter. If you need to know how to install NxFilter read the first section of this tutorial.

But unlike NxFilter after you install it you can't use it as your DNS server right away. This is because NxFilter-Cloud is a multi-tenacy program for commercial service. You're not supposed to use it for your internal network. Your clients use it for their network. So you need to create an account for your client first.

On NxFilter-Cloud there are 3 kinds of users.

admin > operator > user

'admin' is you and 'operator' is your client and 'user' is the user in your client's network. So you need to create an operator first. To create an operator login to NxFilter-Cloud GUI as admin and then go to 'Operator' menu. You can create an operator there.

When you create an operator there will be a default user and default policy will be created for him/her with the same name as the operator. And the default password for an operator is also same as the name of the operator. Once you create an operator you can login with the operator account to setup a user for testing. Normally you'd need to associate your IP address to the default user for testing.

- Go index -
Differences from running NxFilter
1. Authentication enabled always
You don't want to make your service available to anybody. You want to service it to your clients only. so there's authentication enabled by default. Another reason for enabling authentication is that your NxFilter-Cloud can be a target of DDOS attack. So you need to service it only for the known clients.

2. Login redirection disabled at default
You still can use password based login with NxFilter-Cloud but if you use that on public network you can be a target of DDOS attack. You'd better disable it on public network. When you disable it NxFilter-Cloud doesn't respond to any unknown user or client. So you'd need to use login redirection or login-page only when you service NxFilter-Cloud in a private network or on VPN.

3. Magic password for accessing operator GUI
As an administrator of NxFilter-Cloud sometimes you would need to access operator GUI for technical support purpose. For that reason NxFilter-Cound has one more passowrd for admin. It is called as magic password. With this password you can access any operator's GUI. The default magic password is 'magic1023' and you can chage the password on 'Config > admin'.

- Go index -
Building your own billing system for NxFilter-Cloud
Building your own billing system for NxFilter-Cloud When you service NxFilter-Cloud commercially you want to have an automated billing system or you want to create and manage your client account on your side. As all the GUI layer exposed as JSP pages it's not that difficult for you to build your own builling system with NxFilter-Cloud.

To build your own billing system you need to be able to create, edit an operator which is your client account on your own webpages. Suppose if you need to create an operator with these properties.

- Name : triton
- Password : triton1234
- Email : tmail0487@yahoo.com
- Max user : 3
- Max user IP : 3
- Max policy : 3
- Max whitelist : 20
- Max free-time : 10

The JSP code would look like below.

<%
OperatorData data = new OperatorData();
data.name = ”triton”;
data.passwd = ”triton1234”;
data.email = ”tmail0487@yahoo.com”;
data.max_user = 3;
data.max_user_ip = 3;
data.max_policy = 3;
data.max_whitelist = 20;
data.max_free_time = 10;

OperatorDao dao = new OperatorDao();
dao.insert(data);
%>

If you need to update the properties of an operator.

<%
OperatorDao dao = new OperatorDao();

OperatorData data = dao.select_one_by_name(”triton”);
data.max_user = 5;
data.max_user_ip = 5;
data.max_policy = 5;
dao.update(data);
%>

If you need to suspend an operator.

<%
OperatorDao dao = new OperatorDao();

OperatorData data = dao.select_one_by_name(”triton”);
data.stop_flag = true;
dao.update(data);
%>

- Go index -
Clustering with NxFilter
NxFilter has built-in clustering for load balancing and fail-safe. Once you have a master node you can add up to 4 slave nodes to your cluster. All the slave nodes in your cluster sharing the setup from the master node. So you can control everything on your master node.

Creating a cluster

To create a cluster firstly you need to make a master node. On 'Config > cluster' you can make your NxFilter to be a master node. And then you can add the other NxFilter installation as the slave nodes to your mater node. You need to restart NxFilter after change cluster setup.

Starting clustered NxFilter

When you start NxFilter cluster you need to start the master node first and then you can start the slave nodes. This is because your slave nodes need to download the initial setup from the master node when they start.

* If your slave node doesn't work properly there might be a connectivity issue between your cluster nodes. TCP 19003, 19004 ports need to be opened for you cluster nodes to communicate with each other.

Load balancing and fail-safe

One good thing about dns-filter is that there's already a method of load balancing and fail-safe existing. Make your clustered NxFilter to be the primary and secondary DNS servers for your client PC. Then you get the load balancing and fail-safe.

* If you want to have load balancing and fail-safe for your block-page and login-page or policy update for your NxLogon and NxClient agents you need to set multiple block redirection IP separated by commans on 'Config > config'.

When a cluster node down

When a slave node down the other slave nodes and master node will not be affected. The rest of the nodes will be working normally. But when your master node down you lose filtering but DNS lookup still working. When the master node restored your slave nodes re-establish the connection to the master node automatically.

If you setup the alert email on 'Config > config' you will receive an email for cluster node down.

Access control for slave nodes

If you add all your slave node IP addresses into 'Config > cluster' on the GUI of your master node any unregistered IP address will be blocked from the master node.

Using a different set of resolving DNS servers for a slave node

You might want to have a fail-safe measure for the connection to your resolving DNS servers by having a different set up resolving DNS servers for your slave node. But the problem is that in a cluster every node share a master node's configuration. So you can't do it using GUI. However you can use '/nxfilter/conf/cfg.properties' for that purpose. There's 'upstream_dns' param you can add into the file. For example if you want to use '192.168.0.100' and '192.168.0.101' as your upstream servers you can add this line into 'cfg.properties' file.

    upstream_dns = 192.168.0.100,192.168.0.101

Monitoring slave node state

You can view connection state from your slave nodes on 'Config > cluster'. Once you setup your cluster then your slave nodes will appear with the last contact time on the page. It is also showing request, block, user, client-ip count information. These counter information will be set to 0 on midnight or when you restart NxFilter as they are being stored in memory.

- Go index -
Bandwidth control with NxFilter
NxFilter supports per-user based bandwidth control. The idea is simple. Using Netflow data NxFilter associates the traffic data to its user login session and if there's a user consumed bandwidth over the limit NxFilter blocks all the DNS requests from the user.

One good thing is that it is not just about HTTP traffic. Since NxFilter uses NetFlow data you can monitor and block other protocols including HTTP, FTP, torrent, streaming , IM, Skype and any other protocols working on TCP/UDP.

To enable bandwidth control you need to have a router or firewall supporting NetFlow version 5 in your network. If you have NetFlow v5 supporting router setup it sending NetFlow data to NxFilter. And then run the built-in NetFlow collector of NxFilter. You can run the collector on 'Config > config > NetFlow'. Then you can setup per-policy bandwidth limit.

There are several rules for NxFilter importing NetFlow data. Firstly the source or destination IP addresses of a NetFlow data need to be associated to one of the IP addresses of logged-in users of NxFilter. Secondly NxFilter ignores internal traffic. This means one of the source or destination IP addresses needs to be a public IP. This is because you are only interested in inboud or outbound traffic from/to the Internet. And lastly NxFilter keeps only TCP/UDP data.

* When you have a cluster there might be up to 1 minute delay for the update of 'blocked user list by bandwidth'. This is for preventing too much frequent communication between the master and slavess.

* Currently NxFilter supports NetFlow v5 only.

- Go index -
Detecting and preventing malware/botnet activity with NxFilter
One of the features of NxFilter is being able to detect and block malware/botnet activity by analyzing DNS packet. In reality malwares and botnets are another form of network clients or server programs. This means they are also heavily relying on DNS protocol to find their master servers or peers to communicate with each other.

For example if you have a spambot in your network the spambot will make a lot of DNS queries for MX records for target domains to send emails. But normally your client PC doesn't need to make MX queries unless they have an email server running on it.

Another example would be the botnets using the 'TXT' record or other DNS records as their communication tool. These are the real world examples of malwares using DNS protocol as their communication tool.

ex1) Trojan.Spachanel was using SPF record.
ex2) W32.Morto was using TXT record.

The other method we can think of would be detecting abnormal length of domain. When we tested top 100,000 domains from www.alexa.com all the domains except 142 domains were shorter than 30 characters. But there are abnormal domains trying to look like some URL of target domain. This is an example of www.phishtank.com which is trying to look like a webpage of www.ebay.co.uk but actually it is a phishing domain.

ex1) cgi.ebay.co.uk-item-css.ebay-motors.session.id-sj3mzbasf3k12z581668115.login-wpadmin-sw.buyitnow.sign-in.secure-process657943sddh53zix34235hj65rj.xml.config-page.overview.buyer-protection-jsp.wpcs.spiridus-magic.org

So detecting botnet/malware by analyzing DNS packet would be one of the effective technique we can think of. NxFilter provides these blocking options on its policy setup.

- Max domain length
- Block covert channel
- Block mailer worm
- Block DNS rebinding
- Allow 'A' record only

But you can say that the most effective way of preventing malware/botnet in your network would be allowing only 'A' record query from your client PC. In most cases your client PC doesn't need to make a DNS query for any records except 'A', 'AAAA', 'PTR' , 'CNAME'. If you have some email servers or the other network servers in your network then you can apply a different policy for them or bypass them from NxFilter.

- Go index -
Removing embedded adverts in webpages
As of v2.2.7 we added Javascript code hiding embedded block-page into the default block-page. If you updated it from an older version use 'restore-default' button to enable it.

There are webpages having embedded adverts from other domains. One of the problems for blocking these adverts with NxFilter would be having a mangled webpage as the result of blocking. Your block-page replaces the embedded adverts.

To avoid of having this kind of problem there are 2 ways of removing embedded adverts with NxFilter. One is using a special category in 'Category > custom' which is called 'ad-remove'. If you add some domain into this category and you block the category somewhere NxFilter blocks the category with blank block-page.

The other method is to block it using the 'Ad-remove' option on a policy. With this option NxFilter blocks 'adv' category of Shallalist. If you want to use this option you need to import Shallalist first.

* After you add a domain into 'ad-remove' category you need to block it on whitelist or policy otherwise it will not be blocked.

- Go index -
Syslog exportation
NxFilter provides syslog exportation function. The exported data is a character string that you can split by '|'. For example if you have the following syslog data.

2013-01-28 10:53:23|Y|www.bbc.co.uk|pwuser|192.168.0.101|admin|news|Blocked by admin|33

It can be parsed into these values.

- Date : '2013-01-28 10:53:23'
- Block yes/no : Y
- Domain : www.bbc.co.uk
- User : pwuser
- Client IP : 192.168.0.101
- Policy : admin
- Category : news
- Blocked reason : 'Blocked by admin'
- DNS query type : 33

- Go index -
Performance tuning guide
NxFilter was designed to handle more than several thousands users. Currently, 2014-06-21 the biggest site has more than 7,000 users and they are using 2 NxFilter clustered. I believe you don't need to have high-end machine to see this kind of performance. Just ordinary PC hardware will do the job.

However this does not mean that you get the best performance without proper system requirements. If you don't have enough resource available while having too many users you will end up having slow response from NxFilter. To get the best performance you can adjust several factors for NxFilter.

Memory size

At default NxFilter uses up to 512M of RAM. This is enough for most users. But if you allocate more memory to NxFilter you can expect better performance. In NxFilter startup scripts, '/nxfilter/bin/startup.bat' you have the following line.

    java -Djava.net.preferIPv4Stack=true -Xmx512m -cp "%NX_HOME%"\nxd.jar;"%NX_HOME%"\lib\*; nxd.Main

If you want to increase it to 1G then change '-Xmx512m' to '-Xmx1024m'. When you change this value you also need to think about the redundant memory for other programs on your system. So if you allocate 1G of memory to NxFilter you'd better have more than 1.5G of memory on your system at least.

Disk space and reducing the amount of log data

NxFilter has some fancy reporting feature. You can view all the logging data and daily, weekly report and per-user report. However this kind of reporting consumes a lot of disk space. When you have bigger size of reporting data your system might have some performance degrading.

If you have more than several hundreds users you'd better have at least 10G of disk space for traffic DB. Another option is to reduce the amount of data. To reduce the amount of traffic data you can adjust the value for 'Log retention days' on 'Config > config'. The default value is '30'. It means NxFilter keeps its log data for 30 days.

The other way of reducing the amount of traffic data would be using whitelist with 'Bypass logging' option. For example you can bypass 'e3191.dscc.akamaiedge.net' or 'us-courier.push-apple.com.akadns.net' domains for logging when you have too many requests for these domains from Apple devices.

Use client cache for DNS response

NxFilter manipulates DNS cache TTL to clear out the client DNS as soon as possible to avoid of having confusion from the client side DNS cache but basically this is not a critically needed function and it increases the number of DNS query from your client.

You can set 'Max client cache TTL' in 'Config > config' to '0' to turn off the function. When you turn it off your client PC will not make DNS query against the domain already existing in its cache and it will reduce the load for NxFilter significantly. When you have more than 1,000 users we recommend you to turn off this function.

Increase the number of request handler

NxFilter is a multi-threaded program. It has worker threads handling client DNS requests. The default number of request handler is 4 and it's enough for most cases. But if you think your NxFilter responding slowly you can try to increase it. To increase it to 8 add the following line into '/nxfilter/conf/cfg.properties' and restart NxFilter.

    rh_num = 8

Using local recursive DNS server

One of the possible cause of performance degrading for NxFilter would be its lacking of recursive query. This is not the case when you have just several hundreds users as NxFilter has its own caching. But if you have several thousands users or if you service it over cloud this could be an issue. So we added local recursive DNS option.

However this doesn't mean that NxFilter does recursive DNS query by itself. Rather you can install a recursive DNS server into same server with NxFilter and make NxFilter uses it as its upstream DNS server. So now we have 'local_resolver_port' option on cfg.properties file. If you install something like MaraDNS's Deadwood recursive DNS server and set it to use 10053 port and listening '127.0.0.1' then you need to add this line into cfg.properties file.

    local_resolver_port = 10053

And then restart NxFilter.

- Go index -
GUI customization
The GUI layer of NxFilter was designed for easy customization. It is completely separated from its core part. And it has a naming convention corresponding to its menu structure so that you can find out the file you need to modify easily. For example if you want to modify 'Policy > free-time' on NxFilter menu the file you need to edit is '/nxfilter/webapps/policy,free_time.jsp'.

* In NxFilter-Cloud's case it has operator specific menu. If a JSP file is for operator specific menu then it has 'zop' prefix.

    ex) zop,policy,free_time.jsp

Insert, delete, update, select data

On typical web programming, dealing with DB is almost everything. We are using 'data access object' and 'data object' concept for manipulating data and separating GUI layer. So if we are dealing with 'whitelist,domain.jsp' there will be 'WhitelistDomainDao' and 'WhitelistData' classes.

To insert a new data,

<%
WhitelistDomainDao dao = new WhitelistDomainDao();

WhitelistData data = new WhitelistData();
data.domain = "*.nxfilter.org";
data.bypass_auth = true;
data.bypass_filter = true;

dao.insert(data);
%>

To delete a data when its id is 12,

<%
WhitelistDomainDao dao = new WhitelistDomainDao();
dao.delete(12);
%>

To select a data when its id is 12,

<%
WhitelistDomainDao dao = new WhitelistDomainDao();
WhitelistData data = dao.select_one(12);
%>

And to update the selected data,

<%
data.bypass_filter = false;
dao.update(data);
%>

Lastly, to list data.

<%
WhitelistDomainDao dao = new WhitelistDomainDao();
List data_list = dao.select_list();
for(WhitelistData data : data_list){
    out.println(data.domain + "<br>");
}
%>

Separating your customized GUI into another directory

When you customize NxFilter's GUI it is not a good idea to modify original files directly. You'd better keep it for future reference and create another directory under the installaion directory of NxFilter and copy all the files inside '/nxfilter/webapps' into a new directory and then modify these copied files. For this, NxFilter supports 'www_dir' option on /nxfilter/conf/cfg.properties file.

So if you have your own custom GUI in '/nxfilter/myweb' directory. You need to add this line into your cfg.properties file.

    www_dir = myweb

Then restart your NxFilter.

- Go index -
User contributed documents and scripts
- Chad Coccioniti's script to auto-update URLBlacklist on Windows.
     How to auto-update URLBlacklist

- Tutorial for how to install NxFilter on Ubuntu by Carl Miller.
     Install NxFilter on Ubuntu for beginners

- Rob Asher's NxFilter start/stop script for Linux.
Rob Asher sent me a script to start and stop NxFilter with Linux system. I attached Rob's script and you just need to copy it into '/etc/init' directory on your Linux.

Rob said:
I threw together a little upstart script to manager nxfilter a bit easier on linux systems. I'm using it on CentOS 6.5 but it should work on any linux that uses upstart scripts like RHEL/CentOS/Ubuntu and the file paths match the script. Copy the attached file to /etc/init/ and nxfilter will startup and shutdown with the system plus you can control it with commands like "start nxfilter" or "stop nxfilter" or "status nxfilter". download-script

- Rob Asher wrote a single sign-on script from Linux and Mac clients using NxFilter's login API for custom login script.
     NxFilter + OS X and linux login script

- Mark Page wrote some excellent documents and scripts for NxFilter. It's especially useful if you are an advanced user.
    Of Little Consequence

- Rob Asher's JSP page for restarting NxFilter from GUI.
     View posting

- Stewart Sentanoe's 'Simple way to change blocked page GUI'.
     View posting

- Go index -
FAQ
These are frequently asked questions about NxFilter.

Can I use NxFilter commercially?

The commercial use of NxFilter was allowed in old days. But there were several problems we couldn't solve. The biggest problem was the blacklist option. We use Shallalist as the default blacklist for NxFilter. But it is only free when you use it for non-commercial purpose. Even though we don't ship any blacklist with NxFilter package it may cause a licensing issue in future. That's why we can't allow the commercial use of NxFilter.

However, as of v2.4.0 we added Zvelo blacklist option with the aid of one of our OEM partners. It doesn't need to update or import as it's a cloud based blacklist and it's the best in the market as far as we know. To find out more details about Zvelo blacklist read this, Using Zvelo cloud DB

One of the good things of adding Zvelo option is that now we can allow the commercial use of NxFilter as long as you buy Zvelo license. We are not going to have any license issue with a blacklist provider. And the quality of Zvelo cloud option is enough to compete against any webfilter in the market. If you want to provide NxFilter as a commercial option to your customer go officially without having license issue. We can assist you.

- Go index -

I can bypass NxFilter by accessing websites using IP address.

There are people saying that DNS filtering is useless as they can access websites using IP address. This is very naive thought and simply not true. In today's Internet environment most websites are running on the virtual host. This means there are multiple websites on one IP address. You can't access these websites without domain. And the other reason is that there are many URLs in a webpage. This is especially true when it comes to big portal sites. Those URLs are based on DNS as well. So if there's no DNS resolving the webpage you're trying to view will not be loaded properly in most cases.

* NxFilter can block IP host in URL with its local proxy agents.

- Go index -

It doesn't get blocked/unblocked right away.

This is most likely from the DNS cache on your system. There are 2 kinds of DNS cache on your PC. One is from your browser and the other from Windows OS. Before the cache expires your policy change for blocking/unblocking will not be working. Both caches expire eventually but you might want to clear out manually. If it is the browser cache you can clear it out by restarting your browser.

If you want clear out your Windows DNS cache use the following command in your command-line console.

ipconfig /flushdns

Normally DNS cache from Windows expires in a day at the maximum. Of course it depends on TTL from DNS record but I didn't see it bigger than 86,400 seconds(1 day) usually. About the browser cache it may take several minutes to get expired. But it will get expired and blocked eventually. So in practice this is not a problem as you don't need to block/unblock a site many times a day.

- Go index -

I still get blocked after I logged in.

It is from your browser cache. The fastest solution to this problem is restarting your browser. Or you can wait until the browser cache expired. But it may take several minutes.

There is a workaround for this problem. You can use 'Login domain' in 'Config > config'. If a user type the login domain the user gets the login-page right away. You can make this domain to be the start page of the browser.

* If you use single sign-on with Active directory you can avoid of having this kind of problem.

- Go index -

How do I force users being filtered by NxFilter?

If you have a firewall in your network it's simple. You just need to block all outgoing 53 UDP/TCP traffic except coming from NxFilter. And then you use DHCP to setup NxFilter to be the DNS server for your network. Now NxFilter became the only DNS server your users can use and their DNS setup to point NxFilter being done automatically.

- Go index -

How NxFilter decide which policy to apply for a user?

You can assign a policy to a user directly. If the user belongs to a group then group-policy overrides user-policy. To this it's simple but the tricky thing is that when import users from Active Directory. In NxFilter you can not assign a user to multiple groups but in Active Directory you can.

To solve this problem I introduced 'priority points' concept. If there are multiple groups and if they have several different policies the policy having highest priority-points will be applied. You can set priority-points by editing policy. You can view which policy being applied with 'test' button in 'User > user'.

- Go index -

What's the quickest way of blocking 'facebook.com'?

Add '*.facebook.com' into 'Whitelist > domain' and check 'admin_block' option. This becomes a global blacklist applied to everyone.

- Go index -

I want to block 'facebook.com' only for students.

Create a user or a group for your students and create a policy for the user or group. And then create a custom category to add '*.facebook.com' into. Then in the policy block the category and assign the policy to the user or the group. If you use Shallalist you can just block 'socialnet' category without creating your custom category.

* This is a user specific blocking you need to enable authentication in 'Config > config' first.

- Go index -

I want to allow sales dept to use the Internet freely in lunch time.

You can create a user or a group for your sale dept and define a free-time in 'Policy > free-time' then you can assign a free-time policy which is lenient to the user or group.

- Go index -

How do I change webserver port?

You can change HTTP/HTTPS listening ports on NxFilter. However when you change HTTP port you will lose block-redirection. It's because when NxFilter redirects user browser there needs to be something waiting for the browser on 80 port. And this is the block-page from NxFilter.

To avoid of having confusion for most users I didn't make it on GUI. You need to modify 2 parameters '/nxfilter/conf/cfg.properties' file.

http_port = 80
https_port = 443

After you change the ports you need to restart NxFilter.

- Go index -

How do I reset admin password?

NxFilter provides '/nxfilter/bin/reset_pw.bat' util scripts to reset the password. Once you run these scripts admin name and password to GUI will be reset to 'admin'. When you execute these scripts NxFilter must be running.

* There is '/nxfilter/bin/reset_acl.bat' to reset access restriction to GUI as well.

- Go index -

Can I bind NxFilter to a specific IP address?

You might want to bind NxFilter to a specific IP address to avoid of having port collision problem. You can bind NxFilter to a specific IP address using 'listen_ip' parameter in '/nxfilter/conf/cfg.properties' file. If you set it to '0.0.0.0' NxFilter will listen to all the IP addresses of the system but if you set it to a specific IP address NxFilter will listen to the specified IP address.

* Even if you bind NxFilter to a specific IP address you can not run multiple NxFilter on the same machine. This is for avoiding of port collision with anotehr DNS server or webserver.

- Go index -

How do I bypass my local domain?

As of v1.6.1 you can bypass queries for your local domain. This is different from whitelist as it gets no filtering and no logging. You can set your local DNS server and local domains and then if there are queries for your local domains NxFilter forwards the queries to the local DNS server and bypass the filtering and logging.

There are 2 parameters you can add into '/nxfilter/conf/cfg.properties' file. One is 'local_dns' and the other is 'local_domain'. For 'local_dns' the IP of your local DNS server and for 'local_domain' your local domains which you want to bypass from NxFilter. You can have multiple domains separated by commas.

local_dns = 192.168.0.101
local_domain = rainbowx.local, mydomain.local, yourdomain.com

- Go index -

How do I update Shallalist nightly?

If you want to update Shallalist on midnight you can write your own script and add it into crontab if it's on Unix.

#!/bin/sh /nxfilter/bin/shudown.sh
/nxfilter/bin/update_sh.sh
/nxfilter/bin/startup.sh -d

If you are on Windows use 'net start NxFilter' and 'net stop NxFilter' commands. It would look like below.

net stop NxFilter
c:/nxfilter/bin/update_sh.bat
net start NxFilter

- Go index -

Too many requests for some domains.

One of NxFilter users reported that he gets too many requests for these domains.

e3191.dscc.akamaiedge.net
us-courier.push-apple.com.akadns.net

According to him he has almost 1,200 users in K-12 school environment and they are using only Mac. These domains being used for CDNs for Apple and he gets about 100k of requests for these domains a day. The problem of this kind of situation would be that you can't view the other domains on the logging list or report and as a result it would be difficult for you to understand what's going on in your network by watching logging and reporting of NxFilter. And it will also slow down your NxFilter with the massive amount of the load. To avoid of having this kind of problem what you can do is using whitelist with bypass_logging option. Once you bypass these domains or the domains being used for some special purpose you will have lesser load for NxFilter and lesser amount of log data. * If they are important domains for your network we recommend you to set bypass_authentication, bypass_filter options as well.

- Go index -

Can I use exact matching for log search?

You can use square brackets for exact matching on log search.

    ex) [john], [192.168.0.1]

- Go index -

I want to disable login redirection.

There's no config option for that. However you can achieve what you want by emptying your login-page or just having the message like 'We don't have a login-page for you!' instead of a login form.

- Go index -

Why do I need to re-login after lunch break?

Your login session has been expired. If there is no activity(DNS query) from your PC for certain amount of time then your login session expires. You can increase IP session TTL in 'Config > config'.

* If you use single sign-on with Active directory you can avoid of having this problem.

- Go index -