NxFilter Tutorial
1. Getting started

2. Blacklist and domain categorization

3. Authentication

4. GUI overview

6. Working with agent

7. NxCloud

8. GUI customization

9. Misc

9. FAQ
System requirements
- Windows, Linux, FreeBSD or other OS having Java(JRE) 1.7 or higher installed.
- 512 mega bytes of system memory.
- 4 giga bytes of disk space.
- 53 port on UDP, 80 and 443 port on TCP.

* You can run NxFilter on even lesser hardware when you have small number of users but we recommend you to have more than 1GB of memory and 40GB of disk space when you have more than 1,000 users. At default NxFilter uses up to 512MB memory. You can increase the limit of memory allocation to NxFilter in its startup scripts. In '/nxfilter/bin/startup.bat' or '/nxfilter/bin/startup.sh' you can modify the value of '-Xmx512m'.

- Go index -
Install NxFilter on Windows
NxFilter provides a Windows installer. When you download and run 'nxfilter-x.x.x.exe' you will view the following display.

After you follow several steps on the installer, it will try to create NxFilter service. If you see the following message you have NxFilter successfully installed.

To access admin GUI, start your browser and type 'http://localhost/admin' into the address bar. Or if you created a shortcut for web-admin in the installation process you can click it. If you see the following login screen your NxFilter is up and running. The initial admin name and password is 'admin' and 'admin'.

The next thing you need to do would be updating the blacklist of NxFilter. You can follow the instruction from What is a blacklist?

- Go index -
Install NxFilter on Linux
When you install NxFilter on Linux system.
- You need to have root privilege.
- Make sure that your system has Java 1.7 or higher installed.
- You need to make the script files excutable using 'chmod +x /nxfilter/bin/*.sh'.
- You can start NxFilter as a daemon use '-d' option for 'startup.sh'.

1. Download 'nxfilter-x.x.x.zip' file from www.nxfilter.org.

2. Extract the zip file into '/nxfilter'.

3. Go to '/nxfilter/bin' and make script files excutable using 'chmod +x *.sh'.

4. Run 'startup.sh'.

5. To access admin GUI, start your browser and type 'http://your-nxfilter-ip/admin' into the address bar. If you install it on '192.168.0.100' type 'http://192.168.0.100/admin'. The initial admin name and password is 'admin' and 'admin'.

* You might want to start NxFilter automatically when your system startup. On our Ubuntu system we have '/nxfilter/bin/startup.sh -d' in '/etc/rc.local' script. You need to use '-d' option for running NxFilter as a daemon.

- Go index -
Install NxFilter on Ubuntu
As of v2.7.2 we have a 'deb' package for Ubuntu Linux. It installs an upstart script for starting NxFilter at system boot time as well so you don't need use '/etc/rc.local' script anymore. If you install nxfilter-2.7.2.deb on Ubuntu Linux.

To install it, download it from our download page or fetch it using 'wget', install Java and then install NxFilter package using 'dpkg'. Then start it from the upstart script which is installed with the package.

wget http://www.nxfilter.org/download/nxfilter-2.7.2.deb
sudo apt-get install openjdk-7-jre
sudo dpkg -i nxfilter-2.7.2.deb
sudo start nxfilter

After the install to access admin GUI, start your browser and type 'http://your-nxfilter-ip/admin' into the address bar. If you install it on '192.168.0.100' type 'http://192.168.0.100/admin'. The initial admin name and password is 'admin' and 'admin'.

When you update NxFilter installed using 'deb' package and if you update it to v2.7.3 use following commands,

sudo stop nxfilter
sudo dpkg -i nxfilter-2.7.3.deb
sudo start nxfilter

- Go index -
Install NxFilter on Windows manually
This is about how to install NxFilter on Windows manually using 'zip' package. You also can make it as a Windows service using a batch script included in the package.

1. Download 'nxfilter-x.x.x.zip' file from www.nxfilter.org.

2. Extract the zip file into 'c:/nxfilter'.

3. Go to 'c:/nxfilter/bin''.

4. Run 'startup.bat'.

* If you want to install NxFilter as a service run 'c:/nxfilter/bin/instsvc.bat'. It will create NxFilter service. When you unstall it run 'c:/nxfilter/bin/unstsvc.bat'.

* To run NxFilter as a service 'net start NxFilter'. To stop it 'net stop NxFilter'.

* Use 'net start NxCloud' and 'net stop NxCloud' for NxCloud.

- Go index -
Install NxFAdmin by Rob Asher
Rob Asher developed a custom GUI for NxFilter and many people prefer using it over the default GUI of NxFilter. To apply it on NxFilter download it from NxFAdmin repository on BitBucket first.

After you download it, create '/nxfilter/skins' directory and extract NxFAdmin package into '/nxfilter/skins' directory. You many have '/nxfilter/skins/nxfadmin-1.3' contains the newest NxFAdmin package. Then add this line into '/nxfilter/conf/cfg.properties' file and restart NxFilter.

www_dir = skins/nxfadmin-1.3

- Go index -
Start and stop NxFilter
There are several utility scripts for NxFilter in '/nxfilter/bin' directory.

- To start NxFilter : startup.sh
- To stop NxFilter : shutdown.sh
- To see if it is running : ping.sh

On Windows you can use '.bat' files instead of '.sh' files.

* If you installed it on Windows using Windows Installer you probably have it as a service. To start and stop NxFilter service use 'net start NxFilter' and 'net stop NxFilter'.

* Use 'net start NxCloud' and 'net stop NxCloud' for NxCloud.

- Go index -
Client DNS setup
After you install NxFilter you want to monitor and filter Internet activity in your network. To monitor/filter Internet activity you need to make NxFilter to be the DNS server for your network. Since NxFilter is basically a forwarding DNS server you can use the same way of setting up a DNS server for user systems.

The simplest way of setting a DNS server for your users would be modifying the network setup on OS level like the above. But you don't want to setup all the PC in your network one by one. So the best way would be using DHCP server. You just need to modify DNS server address on your DHCP server setup and then your users will be using NxFilter as their DNS server after they get their IP addresses from the DHCP server.

If you have a firewall you can force users to use NxFilter as their DNS server by blocking outgoing traffic on 53 UDP/TCP port. Now NxFilter became the only DNS server your users can use.

- Go index -
When NxFilter not starting
When you find your NxFilter not starting, the first thing you need to do is checking '/nxfilter/log/nxfilter.log' file You can find some information about the cause of the problem. The other things you need to check out would be the port collision and Java installation. NxFilter uses 80, 443 on TCP and 53 on UDP. This means NxFilter itself is a DNS server and a webserver. So if you have another DNS server or webserver running on the same system NxFilter will not start.

About the Java installation, if you use the Windows installer in most cases you will not have the problem but if you install NxFilter manually or if you start it manually not using Windows service you might have some Java related problems. To avoid of having this kind of problem there should be Java installed on the system and you need to have proper environment variables for Java.

If your are on Windows system. You will see this kind of message on the command prompt when you type 'java' if you have properly configured Java.

On Windows system you can set these environment variables.

JAVA_HOME = C:\Program Files\Java\jre7
PATH = %JAVA_HOME%\bin;C:\bin

If it's on Linux NxFilter firstly looks for java in '/usr/bin' and then '/usr/local/bin' so if you don't have java in these directories you need to modify the script files in '/nxfilter/bin' directory or you need to include the path into the environment variables for your system.

To setup 'PATH' system variable for Java you can follow the instruction from the link below.

    - http://java.com/en/download/help/path.xml

- Go index -
What is a blacklist?
A blacklist is a database of categorized domains. This is an essential part of a webfilter for blocking websites by categories. NxFilter supports the following blacklists.

1. Shallalist
Free for non-commercial use. It has 1.5 million domains classified into 74 categories. It is maintained on www.shallalist.de. Auto-update tool supported.

2. Komodia
It has more than 22 million domains classified and does dynamic classification. Many commercial webfilters are using Komodia DB. NxFilter uses its cloud option so it doesn't require import or update.

* You can add domains into system categories directly as of v2.8.0. This is useful when you have mis-categorized domains or unclassified domains.

- Go index -
Updating Shallalist
Updating Shallalist is very easy since NxFilter provides an auto-download and update script. To update Shallalist stop NxFilter and run '/nxfilter/bin/update_sh.bat'. NxFilter firstly checks if there's any update from www.shallalist.de and tries to download the blacklist file and then load it into NxFilter DB. Depending on Internet speed It may take several minutes to finish the whole process.

If you need to update it manually download http://www.shallalist.de/Downloads/shallalist.tar.gz and extract it into '/nxfilter/shallalist1/BL' then run 'update_sh.bat /nxfilter/shallalist1/BL' command.

cd /nxfilter/bin
update_sh.sh /nxfilter/shallalist1/BL

- Go index -
Using Komodia cloud service
To be compared to those expensive commercial webfilters the only weakness of NxFilter is its relatively smaller blacklist DB. When we use Shallalist, even though it has 1.5 million domains categorized, it is just not enough to cover today's huge Internet. As a result we see a lot of 'unclassified' domains on NxFilter. This means our filtering by category can't be the best.

To address this issue we have been trying to include a commercial blacklist option and finally we made a partnership with Komodia. It has more than 22 million domains classified already and it does dynamic classification. Its DB size is growing. When we think about its quality and pricing Komodia is the best option we could find so far.

* We use a customized category set for NxFilter. It has 68 categories.

Pricing plan

The pricing is 3 USD per-user, per-year. Before you buy it you can have 14 days trial. There are non-profit organization discount and volume discount. We also have an unlimited license option starting from 10,000 users. If you want to find out more about these pricing options contact 'support @ nxfilter.org' first.

* We are recruiting resellers for our commercial options. If you want to become a reseller contact us using 'support @ nxfilter.org'.

Activation of the license

After you purchase Komodia license you will receive 'license.lic' file. You need to copy it into '/nxfilter/conf' directory. And then select Komodia option on 'Category > System' and restart NxFilter.

Counting the number of users

NxFilter counts the number of logged-in users or client IP addresses on daily basis. If either of them exceeds the specified user number on your license any unlicensed users will be appeared as blocked on your log view. And those domains queried by the unlicensed users will be categorized as 'Unclassified'. However since it is a warning measure this blocking is not actually happening on user side.

* To find out the number of users in your network, use daily report on 'Report > Daily' or the usage report for last 30 days on 'Report > Usage'.

- Go index -
Komodia categories for NxFilter
When we use Komodia DB we use a custom category set designed for NxFilter. These are the 69 categories we defined.

Abortion Abortion and pregnancy
Ads Sites that serve ads
Adult Adult content graphically or in text including semi nude
Alcohol/Tobacco Alcohol and tobacco
Astrology Astrology and various cold reading methods
Blog Blog sites
Business/Service Business and services including ISP, web hosting, Mobile, Cable, TV
CDN Sites that are hosting content and data for other sites, including images and videos
Chat Online chatting or chatting applications
Classified Classifieds sites, second hand sales and services
Computer/Technology General computer and Internet sites, technical information
Cooking/Food Cooking, food, restaurants
Crime/Illegal Crime instructions or illegal products like fake ID and diplomas
Dating Dating and relationship
Depress Obituaries, grief and loss
Download Software download
Drugs Illegal drugs
Education Schools, kindergartens, colleges, universities
Entertainment/Arts Books, comics, movie reviews, actors and models, celebrity gossips, arts, paintings
Fashion/Beauty Clothing and fashion, life style, beauty, personal care and grooming
File hosting Sites that allow users to upload and host files
Finance/Real Estate Bank, insurance, real estate, bitcoin
Forum Forum
Gambling Gambling
Game Games and toys including online games
Government Government owned sites
Hacking Hacking and cracking information
Hate/Racism Hate crime and racism
Health/Medical Medical and health information sites and online pharmacies
Home/Gardening Home improvements, gardening, decorating and DIY
Humor/Nonsense Humor, jokes, nonsense and time waste
Hunting/Fishing Hunting and fishing
Job Job search and offers
Kids Sites for kids
Knowledge/Learning Science, historical events, person, general knowledge and online courses
Legal Lawyer and low firms and legal advice and tips
Local City information, state information, law enforcement
Military Military sites with .mil or military related
Misc General websites, not porn
Music Music, lyrics, bands, musical instruments and downloading songs/mp3
Nature Nature and environment
News/Magazine News, magazines, weather
Pharmacy Online pharmacies
Phishing/Malware Phishing, spam, scam, virus infected sites
Politics Political opinion and news
Porn Pornographic content
Portal Portal sites, bookmarks
Proxy/Anonymizer Sites that contain proxy information to bypass filtering
Recreation/Hobby Hobbies and recreations, pets and animals
Reference Dictionary, map, translation, IP lookup
Religion Religious sites including sects and cults
School Cheating Ready made works for students, or school cheating
Search Engine Search engines
Sex Education Talking about sex with educational purpose
Shopping Shopping and online auction
Social Network Social networking service
Sports Sports and martial arts
Streaming Media Online video or audio and downloading movies
Surveys/Polls Surveys and polls services
Tracker Tracking user behavior, web analytics
Travel Travel, vacations and holidays
URL Shortener URL redirection or shortners
Vehicle Vehicles including cars, boats, planes, bikes
Victim Help Sites for helping victims of crime
Violence Violent content, suicide and profanity
Warez Illegal download sites or torrents
Weapon Selling or talking about weapons including guns, knives
Webmail Webmail service

- Go index -
NxFilter and authentication
NxFilter provides several authentication methods including Active Directory integration or single sign-on.

Why authentication

When you install NxFilter first time you only have one policy and it applies to all the users in your network. But what if you are working for a school as a sys-admin and you want to apply different policies to students and teachers. For students more strict policy and for teachers bit lenient policy. Now you need to differentiate users. That's when you need to enable authentication.

Which authentication

NxFilter supports several ways of authentication. You can choose one of them or mix and match some of them.

1. IP based authentication
This is the simplest form of authentication. When you use static IP address for your client PC this might be the best choice. Just associate the IP address of the client PC to the user you create on NxFilter GUI. You also can create a user covers a certain IP range.

* Many people try to use IP based authentication without enabling authentication on 'Config > Setup'. But IP based authentication is still a method of authentication so you must enable it first.

2. Password based authentication
When you enable authentication NxFilter blocks user request with its login-page unless the user already logged in or having IP address associated to him/her. To go through the login-page your users need to enter his/her username and password. You can set this password to a user on NxFilter GUI after you create a user. Means you can have password based authentication without Active Directory or LDAP.

3. LDAP based authentication
If you have OpenLDAP or Active Directory your users can go through the login-page using their LDAP credentials. To use this feature you need to import your users from your LDAP server first.

4. Login-token based authentication
NxFilter has a special concept called 'login-token'. This is used for remote user authentication or filtering. This login-token being created for each user when you create or import users. You can use this login-token for NxClient when you need remote filtering or dynamic IP update.

5. Single sign-on against Active Directory
Many people want to deploy their webfilter transparently. Or you don't want to show any login prompt to your users. So NxFilter provides Active Directory integration. Once you set it up your users don't need to go through login-page and your users will appear on NxFilter GUI with their AD username and group.

- Go index -
Single sign-on with Active Directory using NxLogon
When you have Active Directory you want to have single sign-on against Active Directory and not showing any extra login prompt to users. For this we have a client program that is NxLogon. When you run NxLogon on a user PC it creates and keep a user login session on NxFilter.

However you don't want to install and manage this program on every PC in your network. So we use the logon script of GPO(Group Policy Object) on Active Directory. This logon script is being executed whenever a user logon to Active Directory domain. And we can write our own logon script to launch NxLogon and then we get the single sign-on.

* If you want to have single sign-on against Active Directory you firstly need to import users and groups from your Active Directory. To import users and groups read, GUI - User

* NxLogon uses TCP 19002 port to talk to NxFilter.

After you launch NxLogon it will create a login session for the Windows user account and it will refresh the session on every minutes. You can follow these steps to launch NxLogon from GPO.

1. Download nxlogon-4.x.zip package from www.nxfilter.org.

2. Modify IP address in 'nxlogon.bat' to point NxFilter. If you use clustering add multiple IP addresses separated by spaces.

3. Open 'Administrative Tools > Active Directory Users and Computers' on your DC.

4. Open 'Group Policy' tab in properties of your Active Directory Domain.

    

5. Click 'Edit' button and then go to 'User configuration > Windows Settings > Scripts (Logon/Logoff)'.

    

6. Click 'Logon' and click 'Add' and then click 'Browse' button. You will see 'Logon' directory to select file. Copy your 'nxlogon.bat' and 'nxlogon.exe' from NxLogon package into 'Logon' directory. You can drag and drop the files into the directory.

7. Select 'nxlogon.bat' which you copied into 'Logon' directory as the logon script to add.

    

8. Now every time users logon to their systems 'logon.bat' will be executed and it will launch 'nxlogon.exe'. You can check the process running on Windows task manager.

    

* Since NxLogon is running background you can't see it running. If you want to check if it's running see 'Processes' tab on 'Windows Task Manager'.

* If you want to remove the login session immediately after user logout use 'nxlogoff.bat' as a logoff script in GPO.

* When you run multiple instances of NxLogon on the same PC by multiple users it would cause a mess for user detection process. Your users might be appeared with several different names on log-view. To block multiple instances on the same system use '-bm' option.

* Rob Asher wrote a single sign-on script from Linux and Mac clients using NxFilter's login API for custom login script.
     NxFilter + OS X and linux login script

- Go index -
Single sign-on with Active Directory using NxMapper
While using NxLogon is still the best solution for AD single sign-on but some people find it difficult to setup all these GPO and logon script for launching NxLogon. So we offer an easier way of implementing single sign-on against Active Directory. When you install NxMapper on your domain controller it will grab username and IP address pair and send it to NxFilter.

* If you want to have single sign-on against Active Directory you firstly need to import users and groups from your Active Directory. To import users and groups read, GUI - User

Install and run NxMapper

We offer a Windows installer for NxMapper. It will install NxMapper as a Windows service. After you install NxMapper using the installer you will have its setup program running.

* NxMapper needs to be installed on a domain controller.

* You can add multiple IP addresses separated by commas if you run a cluster of NxFilter.

* NxMapper uses TCP 19002 port to talk to NxFilter.

After you modify the config value test your setup first and then start it.

Differences from using NxLogon

Although it's a lot easier compared to using NxLogon, NxMapper also has its own limits. Firstly you can't use the application control function from NxLogon.

The other thing is that the login session can be expired. While NxLogon refreshes the login session on evey minutes, NxMapper creates or refreshes user login session only when there's user activity on DC. So if your users don't use Internet for a while then their session will be expired. Once the session expired your users will be redirected to the login-page of NxFilter.

To prevent the expiration of the login session you can increase the session timeout value on 'Config > Setup > Block and Authentication > Login session TTL' on the GUI of NxFilter. If you increase it to 120 minutes it will be enough to cover the lunch break. After your users resume using the Internet then the session continues.

Terminal server exclusion

When we use NxMapper we might have some problem with terminal server. If there are multiple users on one system the IP address of the system will be associated to the user whose action detected lastly by NxMapper. It means your users can be appeared on NxFilter with a different username. To prevent having this kind of problem the best solution would be creating an IP based user for your terminal server.

- Go index -
Single sign-on with Active Directory, OpenLDAP using NxClient
NxClient is basically a remote user filtering client for mobile workers with their own laptop. But you can use it for single sign-on against Active Directory or OpenLDAP. One good thing is that since there's Mac OS version of NxClient you can have single sign-on from Mac OS.

If you already tried to use NxClient you already know that single sign-on using NxClient is possible with its 'login-token' concept. But with this approach the problem is that it's almost impossible to setup several hundreds of NxClient installations with their own 'login-token'.

So we provide a way of running NxClient on local network without setting up different login-token to each client PC. What you need to do is to install NxClient using a common login-token for all the client PC. Then when it starts it will look for its server that is NxFilter on local network and if it finds one it will create a login session for the current logged-in user or console username.

* For NxClient being able to detect local NxFilter, you have to use NxFilter as the DNS server for your client PC.

Another good thing is that since it's running as a Windows service or a daemon on Mac OS X your users can't stop it.

To find out more details about NxClient read this, NxClient and remote user filtering

- Go index -
Custom login script for single sign-on
Currently NxFilter supports single sign-on with Active Directory. However some people need more than that. You might want to have single sign-on from your Linux clients against Active Directory or you might want to have single sign-on with your OpenLDAP users.

NxFilter supports an API set for creating IP session through HTTP protocol or webpage. You need to write your custom login script to call some webpage on NxFilter's built-in webserver. And then your users don't need to see the login-page.

Currently it's on '/nxfilter/example/login_user.jsp'. Initially the access of the page is restricted to localhost only for security reason but you can edit the JSP page to allow calls from your local network.

You can call the webpage this way.

    http://192.168.0.100/example/login_user.jsp?ip=192.168.0.100&uname=john

As you see 2 parameters being passed. One is the IP address of your user and the other one is the associated username. The username should be imported or created on NxFilter side already.

One thing you need to consider when you write your own login script is that it might be better to call the webpage periodically. There is a session timeout concept in NxFilter. If there's no activity from a logged-in user for certain amount of time the login session will be expired. So if you don't want to show your users the login-page you'd need to refresh the login session periodically.

There are 3 methods of UserLoginDao class for custom login script.

- create_ip_session(String ip, String uname) : Creating login session with the IP and username.
- delete_ip_session(String ip) : Deleting login session with the IP.
- find_user(String ip) : You can find a logged-in username by its associated IP address.

All the example JSP pages are in '/nxfilter/webapps/example' directory.

- Go index -
The order of authentication methods
NxFilter uses multiple authentication methods at the same time. If there is a user request NxFilter tries to find its associated user based on the IP address. Firstly NxFilter looks into the memory map for IP based users. If there's no user associated to the given IP address it looks into IP session map which populated by sigle sign-on agent or user login through NxFilter login-page. And if there's still no user found finally NxFilter redirects the user request to its login-page.

This is the order of authentication methods application.

1. Single IP associated user
You can create an IP based user associated with a single IP or an IP range on NxFilter. We put single IP association first so that you can exclude some systems from IP session or single sign-on by creating an IP based user.

2. IP session
When a user login to NxFilter using login-page or some agent like NxLogon or NxClient it keeps login session based on user IP and refreshes the session as long as there's user activity.

3. IP range associated user
Some people want to allow anonymous or guest users to use Internet while having authentication. In that case you can create a user associated to the IP range that covers your entire network. Now if a user has a login session he/she will be appeared with the associated username but without login he/she will fall into the IP range user that covers your entire network.

As of v2.6.1 we have 'Most specific IP range comes first' rule for ordering among IP range users. If there are overlapped IP ranges the smaller IP range will be applied before the others.

- Go index -
Deployment of NxFilter in Active Directory
We provide several methods of AD integration but some people find them hard to understand. So we want to explain what is AD integration for NxFilter and when to use it and how to use it at conceptual level.

The reason people want to integrate their webfilter into Active Directory is that they want to apply filtering policies based on their AD user and group. And plus they don't want to have their users going through any extra login step to use Internet except the login to their own PC. So for NxFilter 'AD integration' means using the same user account from your Active Directory to differentiate users and having single sign-on from your Active Directory.

Now we know what is AD integration and why we need it. But how to do that? On NxFilter the first thing you need to do for implementing AD integration is importing users and groups from Active Directory. It means you need to let NxFilter be aware of your users and groups. You can do that on 'Users > Active Directory'.

* You don't need to setup zone-transfer for AD integration. It's for compatibility to older versions of NxFilter. NxFilter bypasses AD DDNS queries automatically based on your AD importation setup. Typically you don't have a problem without zone-transfer.

* You need to allow nonsecure dynamic update for NxFilter on your MS DNS zone properties.

After you import your users and groups your users will be able to use their AD credentials on NxFilter's login-page. So we already achieved AD integration to a certain level.

* You have to enable authentication on 'Config > Setup' to see login-page.

However your users don't want to go through any login-page so the next thing you need to do is running single sign-on agent so that NxFilter can create login sessions for your users when they login to their own PC.

When it comes to single sign-on agent there are several choices. NxLogon, NxMapper, NxClient, NxUpdate, NxBlock. You can use just one of them or mix and match them for complementing each other.

* About the differences between these agents read 'Differences between agents' in our tutorial.

Lastly I will give you an example deployment scenario. Suppose we are in a company environment. Many Windows PCs and some Macbooks and recently we bought several Chromebooks. And people bring their own iPhone and Android phones. And plus we have several Linux servers for our own website and file sharing. There are some mobile workers using company laptops. Some are using Windows and some are using Macbooks. And you want to filter all of them whether they are inside office or outside office with their AD accounts.

The first thing you need to do is to setup AD user and group importation. And then use NxLogon for these Windows PCs. It also can do application control and proxy filtering. But NxLogon doesn't work with Macbooks. For these Macbooks you can use NxMapper as we just need to install it on a domain controller. If you have some mass installation way for these Macbooks or have only several Macbooks you can go with NxClient here. It is basically a remote user filtering client for mobile workers but you can use it as a single sign-on agent in local network and it supports Mac OS X.

* You can also use NxUpdate if you have just several Macbooks but it's bit different from AD integration.

And then you want to deal with these mobile workers. You can install NxClient on their laptops. There are Windows and Mac versions for NxClient. And good thing is that when they are out of the office it works as a remote user filtering client but inside office it works as an AD single sign-on agent like NxLogon.

For Chromebook and iOS you can try NxBlock. It's basically a Chrome extension and you can use it as a remote filtering client or single sign-on agent for Active Directory. And for your servers I wouldn't do filtering and setup them with static IP addresses and use another DNS server for them. You don't need to block anything for them normally.

For your Android devices, just let them go through login-page. We don't support them yet.

If you want to avoid of showing login-page absolutely setup a default user and policy for everybody. On NxFilter we have preference ordering among authentication methods. When you setup a user with the IP range which covers your entire network and if there's a user not having login session on NxFilter he/she will fall into the IP range user.

- Go index -
GUI - Config
These are mostly system configuration parameters for NxFilter.
Config > Setup > Block and Authentication

- Block redirection IP
Simply speaking, this is the IP address of NxFilter itself. If there is any blocked DNS request, it will be redirected to this IP address. When you first start NxFilter it will attempt to decide its IP address from your system setting.

* You can add multiple block redirection IP addresses separated by commas for load balancing purpose.

- External redirection IP
When you use remote filtering you might need to use a different 'block redirection IP' for remote filtering clients since they are outside of your network. If you leave this blank NxFilter will use 'block redirection IP' for redirecting remote filtering clients.

- Enable authentication
This option is required for AD-integration or any other user authentication method. After you enable this option any unauthenticated users will be redirected to NxFilter login-page. As a result users will be forced to login to use the Internet.

* One thing you need to know is that even if you use only 'IP based authentication' it is still a method of authentication. So you need to enable authentication. Many users are failed with this.

- Login domain
You can access NxFilter login-page using the domain defined here.

- Logout domain
You can clear out user login session using this domain.

- Login session TTL
NxFilter keeps the user login session after a user logged in through its login-page so that your users don't need to see the login-page again while they are using the Internet. But this login session needs to be expired eventually. It is especially required when there is a shared PC by several users. So we have 'TTL' value for this login session. If a user doesn't use the Internet for the specified amount of time here his/her login session expires and the user needs to re-login.

Config > Setup > DNS Setup

- Resolving DNS server
NxFilter is basically a forwarding DNS server. You need to setup the IP addresses of the DNS servers which resolve the DNS queries forwarded by NxFilter. You can have up to 3 resolving DNS servers.

- Resolving DNS query timeout
Timeout for a DNS query to a resolving DNS server.

- Max client cache TTL
Your client PC has its own DNS cache. This client side cache might cause some problem in DNS filtering. Your blocking doesn't work until the cache expires. To reduce the impact from this kind of problem NxFilter provides an option for manipulating client cache TTL. If you set the value to '60' NxFilter modifies the DNS cache TTL to '60' if the TTL is bigger than 60.

0 - Don't touch it.
60 - Don't touch it if it's smaller than 60 and make it '60' if it's bigger than 60.

We introduced this function to minimize the effect from the client cache. However if you have more than 1,000 users you would better turn this function off to have better performance.

* If you have many users it's better to set 'Max client cache TTL' to 0 as you may have a performance issue from too many requests from your clients.

- Response cache size
NxFilter has its own cache for DNS query result. Once it has a query result from resolving DNS server it keeps it in the cache until it expires by TTL value from the DNS packet and answering clients with the records from the cache. Generally speaking the bigger cache size would be better for the performance. Currently the default size is 100,000 and it's enough for most sites.

Config > Setup > Syslog

Like the other enterprise security software NxFilter supports syslog exportation of its log data. You can build your own reporting system with this feature or you can monitor all the logging in real-time way.

- Syslog host
The host address to which you want to send syslog data.

- Export blocked only
With this option NxFilter sends log data of blocked request only.

- Enable remote logging
Enables syslog exportation.

Config > Setup > NetFlow

NxFilter supports bandwidth control. This is possible by importing NetFlow data using the built-in NetFlow collector.
For more detail read this, Bandwidth control with NxFilter

- Router IP
The IP address of the device sending NetFlow data to NxFilter.

- Listen port
The UDP port number of the NetFlow collectors.

- Run collector
Run NetFlow collector. After change this option you need to restart NxFilter.

Config > Setup > Misc

- Admin domain
You can access admin GUI using IP address of NxFilter or 'localhost'. But once you setup admin domain you can access it using the domain you want. For example if you use 'admin.nxfilter.org' as your admin domain you can access your admin GUI by typing 'http://admin.nxfilter.org/admin' into your browser address bar.

* This only works when you use NxFilter as your DNS server. Otherwise you need to register your admin domain to your own DNS server.

- Bypass Microsoft update
You don't want to block Microsoft update with your filtering. Enabling this option means bypassing 'micfosoft.com' and 'windowsupdate.com' and their subdomains.

- Logging retention period
If you keep your log data too long it will eat up your disk space a lot. You can set how long NxFilter keeps its log data here.

- SSL only to admin page
When you want to allow only HTTPS access to admin GUI enable this option. Once you enable this option you will be redirected to the SSL port automatically even if you use HTTP.

- Auto backup
NxFilter makes a backup file for its config into '/nxfilter/backup' directory on '01:00' everyday. The name of the backup file starts with 'auto-' prefix. You cat have up to 30 backups.

- Agent policy update period
NxFilter supports application control and proxy filtering through NxClient. NxClient fetches the policy for application blocking and proxy filtering according to the period defined here.

Config > Admin

You can change admin name and password for GUI login here.

Config > Alert

NxFilter sends alert email for recent blocking or clustering node down incidents. For example if you want to send an alert email to 'admin @ nxfilter.org' from 'alert200 @ gmail.com' on every 15 minutes then the setup would look like below.

- Admin email : admin @ nxfilter.org
- SMTP host : smtp:gmail.com
- SMTP host : 465
- SMTP SSL : on
- SMTP user : alert200
- SMTP password : ********
- Alert period : Every 15 minutes

Config > Allowed IP

NxFilter has IP based access restriction function for its DNS, GUI, login redirection. You might want to use this feature when you put your NxFilter on a public IP address. You can make whitelist/blckalist way of ACL here.

Config > Backup

You can make a backup for config DB of NxFilter manually. The backup files will be created into '/nxfilter/backup' directory.

Config > Redirection

Domain to IP or domain to domain redirection is possible with NxFilter. It works like custom DNS record.

Config > Zone Transfer

In some situation you need to import DNS zones from the other DNS server. Once you setup zone-transfer here NxFilter imports the DNS zone on every minutes using IXFR protocol.

Config > Block Page

This is the setup for custom block-page, login-page, welcome-page. When you edit your block-page you can use the following variables populated by NxFilter for making your block-page more informative.

- #{domain} : Blocked domain
- #{reason} : Reason for block
- #{user} : Logged-in username
- #{group} : Groups of the logged-in user
- #{policy} : The applied policy
- #{category} : Categories or the blocked domain

Config > Cluster

NxFilter has a built-in clustering. You can make your NxFilter to be a master node or a slave node in a cluster. After you change the values in cluster setup you need to restart your NxFilter to apply new settings.

- Go index -
GUI - User
You can create or import users and groups here. NxFilter supports user importation from Active Directory and OpenLDAP.

Creating a user

You can create a user on 'User & Group > User'. Once you create a user you can edit the user properties on the edit view. There are 3 types of users in NxFilter

1. IP user
It has associated IP addresses or IP ranges and will be authenticated based on IP address.

2. Password user
If you set password for a user it becomes a password-user. You can use the password on the login-page of NxFilter.

3. LDAP user
When you import users from your LDAP servers or Active Directory they become LDAP users. They can use LDAP or Active Directory user credentials on NxFilter login-page.

Properties of a user

- Password : The password for login through NxFilter login-page.
- Work-time policy : The policy being applied during the work-time.
- Free-time policy : The policy being applied during the free-time. You can define a free-time on 'Policy & Rule > Free Time'.
- Expiration date : The expiration date for a user account.
- Login token : The access token for NxClient. It is created when a user created or imported.

Testing a user

When you have LDAP imported users you may have multiple groups and policies for a user. As a result it becomes difficult to figure out which policy a user fall into. This is especially true when you apply different policies for free-time. To find out which is the 'applied policy' for a user use 'test' button on user list. It fetches the information of user state from NxFilter real time way so you get the correct information about how a user being handled by NxFilter.

* You can use this test view to find out how much quota or bandwidth consumed by a user or to reset quota or bandwidth for a user.

Creating a group

You can create a group on 'User & Group > Group'. After you create a group you can setup a policy for the group by editing the group. You also can assign members to a group on edit view.

* As of v2.0.5 you can define group specific free-time on 'User & Group > Group > edit'. When a user belongs to multiple groups and one of the groups fall into free-time range NxFilter applies the free-time policy for the user. NxFilter applies group specific free-time first and then global free-time next.

Importing users and groups from Active Directory or OpenLDAP

You can import users and groups from Active Directory on 'User & Group > Active Directory'. For example if you have Active Directory with the following setup.

- Domain controller : 192.168.0.100
- Domain : nxfilter.local
- Admin username : Administrator

Then you can create Active Directory import setup with the following details.

- Host : 192.168.0.100
- Admin : Administrator@nxfilter.local
- Password : your-password
- Base DN : cn=users,dc=nxfilter,dc=local
- Domain : nxfilter.local

After setup your AD detail you can import users and groups by using 'import' button immediately. You also can setup a periodical import by selecting auto-sync interval.

* When you import users from Active Directory you may have a users belongs to multiple groups and having multiple policies as a result. If you want to select one policy over to the others use 'priority points' property of a policy. Bigger priority points wins over the other policies.

* If you want to find out if there is any issue with the connectivity between NxFilter and your DC, use 'test' button.

* NxFilter also supports LDAP importation from OpenLDAP.

- Go index -
GUI - Policy
With NxFilter you can have multiple policies based on user and group.

Creating a policy

When you install NxFilter there is only one policy that is 'Default'. This policy will be applied to all the users if you don't make any change on NxFilter setup. If you want to apply different policies to different users you need to create other policies and enable authentication.

Editing a policy

After you create a policy you can modify its properties.

- Priority points
If there are multiple policies associated to one user then the policy having the highest points will be applied.

- Enable filter
If you disable this option there will be no blocking from the policy.

- Block all
Block all. Global whitelist overrides this one still.

- Block unclassified
Block uncategorized domains.

- Ad-remove
Block domains in 'adv' category of Shallalist with a blank block-page.

* This is useful when you want to remove embedded adverts without showing NxFilter block-page.

- Max domain length
There are some malwares using domain name itself as a communication tool or a message protocol. These domains are abnormally long while the length of most domains are under 30 characters. You can set a limit for the length of a domain for blocking these abnormal domains. To prevent having false positives NxFilter doesn't apply 'Max domain length' against 100,000 well known domains.

- Block covert channel
Some malwares or botnets are using DNS protocol as their communication tool. They are using DNS query and response to communicate to each other.

- Block mailer worm
Normally you are not supposed to see MX query from your client PC. When NxFilter finds MX type query from your client PC it will be regarded from some malware trying to send emails.

- Block DNS rebinding
When NxFilter finds a private IP address(192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8) on DNS response packet it will be blocked as DNS rebinding attack.

* If you have your own DNS record with private IP address you need to bypass the domain on whitelist.

- Allow 'A' record only
This is the most strict way of filtering malwares and botnets employing DNS protocol as their communication tool. If you are an ordinary office worker you don't need to use any special types of DNS records. With this option NxFilter allows A, AAAA, PTR, CNAME and the other types of DNS records will be blocked. If you need to allow the other types of records from some of your users then you need to apply a different policy to them.

- Quota
NxFilter has quota-time feature. You can allow your users to browse specific sites for certain amount of time. You can set the amount of time to here.

- Quota all
Apply quota to all domains including unclassified domains.

- Blocked categories
You can block user request by categories. These categories are imported from Shallalist or custom categories.

- Quotaed categories
If you check certain categories in 'Quotaed Categories' then your users can access the websites in the categories for the amount of time you specified with 'Quota' above. When a user consumed up his quota his/her DNS requests for those sites will be blocked.

- Safe-search
Enforcing safe-search against Google, Bing, Yahoo search engines and Youtube.

* Yahoo and Youtube safe-search requires local proxy agent filtering.

- Disable application control
Disable application control on policy level.

- Disable proxy filtering
Disable proxy filtering for on policy level.

- Logging only
Monitoring user activity without blocking them.

Define a free-time

You can define a global free-time in 'Policy & Rule > Free Time'. If you assign a free-time policy to users it will be applied during the time defined here. You can have multiple free-time.

* If the start-time is bigger than the end-time then it will break into 'end-time ~ 24:00' and '00:00 ~ start-time' on the same day.

Application control

NxFilter provides application control through its agents, NxLogon and NxClient. For more details read 'Application control' section of this tutorial.

Proxy filtering

NxFilter provides proxy filtering through NxClient. For more details read 'Proxy filtering' section of this tutorial.

- Go index -
GUI - Category
On NxFilter there are system categories and custom categories. System categories are the domain categories defined by your blacklist DB or domain categorization DB. And custom categories are the categories you can create. Once you create a custom category you can add domains into the category. These categories appear on the policy edit view and you can block domains by these categories.

Currently NxFilter supports several blacklist options for system category. If you want to find out more read 'Blacklist and domain categorization' section of this tutorial.

* If you want to include subdomains into a custom category use asterisk.

    ex) *.nxfilter.org

* If you want to find out which category a domain fall into, use 'Category > Domain Test'.

- Go index -
GUI - Whitelist
You can make a global or per-policy whitelist for a domain or keyword here. This one can be used as a blacklist as well by enabling 'admin_block'. If there is a domain having 'bypass_filter' and 'admin_block' options enabled together then 'admin_block' overrides 'bypass_filter'.

- Bypass authentication : When you enable authentication some of your application may not be working. It is because some application needs to access Internet without user attendance. In that case you can try to bypass authentication for the related domains.

- Bypass filtering : When you want to exclude some domains from your filtering policies use this option.

- Bypass logging : Sometimes you find that there are too many log data from some domain which you are not interested in. Or you want to save your disk space by excluding some domains generating too many log data.

- Admin block : This works as a blacklist. When you want to block some domains regardless of your policy use this option.

* When you use whitelist by domain you can use asterisk to include subdomains.

    ex) *.nxfilter.org

* Since the policy association comes after the authentication 'bypass_authentication' in per-policy whitelist will be ignored.

- Go index -
GUI - Main, Logging, Report
NxFilter can keep its log data up to 90 days and you can generate daily, weekly, per-user reports based on this log data.
Main

When you login to your admin GUI you will see the dashboard of NxFilter. There are several charts for showing summary for the last 2 hours. On the bottom of the dashboard you can see 10 recent block logs for the last 12 hours.

* The difference between request-sum and request-cnt is from NxFilter logging system. To reduce the amount of disk access NxFilter keeps all the log data into its memory space. And then it flushes the data once a minute. If there's the same data it only increases the count for the data. This also helps to prevent DDOS attack attempt to NxFilter when you put it on public network.

Logging

You can search user request log with various conditions in 'Logging > Request'. Logging data is being updated once a minute to reduce the load on DB.

In 'Logging > Signal' you can view the log of signals from NxClient.

In 'Logging > NetFlow' you can monitor NetFlow data imported.

* Use square brackets for exact matching on log search.

    ex) [nxfilter], [192.168.0.100]

Report

NxFilter generates daily, weekly, per-user report.

- Go index -
Differences between agents
NxFilter provides several agents. Some are for single sign-on with Active Directory. Some are for remote user filtering and dynamic IP update. Some of them can be used for application control and proxy filtering.

1. NxLogon
Sigle sign-on agent for Active Directory. You can launch it from a logon script on GPO. It supports application control.

2. NxMapper
Another way of Active Directory integration or single sign-on. Unlike NxLogon you install it on a domain controller as a Windows service. It detects user logon on Active Directory and creates login session for the user on NxFilter.

3. NxClient
Remote user filtering agent of NxFilter. When you have a mobile worker or home worker you can install NxClient on their laptop. NxClient will be running as a Windows service and filtering user Internet activity. It supports application control and proxy filtering.

4. NxUpdate
Dynamic IP updater for NxFilter.

5. NxBlock
Remote filtering agent and single sign-on agent for Chromebook or Chrome browser.

- Go index -
NxClient and remote user filtering
NxFilter provides a remote user filtering client software that is NxClient. Once you install NxClient onto user system you can filter and monitor the Internet traffic from the user system on your NxFilter GUI centrally regardless of its location. This means you can filter and monitor Internet activity of your mobile worker or home worker.

* You need to open 53 UDP and 80, 443 TCP ports for NxClient.

Installation of NxClient

After you install it using NxClient installer you will have its setup program running. There are 'Server IP' and 'Login token' parameters and you need to replace their values to your own.

* On NxFilter every user has a login-token. You can find it on 'User & Group > User > edit' on NxFilter-GUI.

* NxClient is a Windows service program. It will start at system startup automatically.

* When you install NxClient or NxUpdate on Mac OS X, read Installing NxClient or NxUpdate on Mac OS X on this tutorial.

After you modify the config value test your setup first and then start it. After starting NxClient you can check if it's working by viewing 'Logging > Signal' on your NxFilter GUI. There will be signals from your client.

* You can add multiple IP addresses separated by commas if you run a cluster of NxFilter.

* To change the config value run 'C:/Program Files/nxclient/setup.exe'.

Signal of NxClient

When it comes to remote user filtering the most tricky part would be how to force users to be filtered. Nobody wants to get filtered and they are away from the office. If they use their personal PC then you can not filter them anyway. But as long as they use a company laptop you still can filter them by installing NxClient on the system.

However what if they uninstall or stop NxClient? NxClient is running as a service and it doesn't provide uninstaller for 'Add/Remove programs' in control panel. So if your users don't have enough privilege to modify their systems you don't have these problems.

But sometimes you need to give your users the 'local administrator' privilege. In that case it's not possible to stop users from uninstalling or stopping NxClient. So we defined several signals with which you can find out what's happening on user system. Once you install NxClient on a system it will send these signals to NxFilter and you can view the log of signals on 'Logging > Signal'.

- START : When NxClient starts it sends START signal to NxFilter.
- STOP : When NxClient stops it sends STOP signal to NxFilter.
- PING : On every 5 minutes NxClient sends PING signal to NxFilter.

You can view these signals on 'Logging > Signal' on NxFilter GUI.

Fail-safe measure for NxClient

You can achieve this by specifying multiple server IP addresses. If one of them fails NxClient talks to the others. However there still can be a time when your NxClient can't connect to all of its server. In that case it bypasses filtering. As soon as the connection restored it will filter again.

Auto-switch between local network and remote network

When you use NxClient on your mobile worker's laptop you might have a problem when they stay in the office with Active Directory integration. Even if you want to apply local network filtering rule based on his AD login username NxClient will be doing its job with its server-ip and login-token.

As a result your mobile worker will be filtered twice. One from NxClient, one from your local NxFilter. And your mobile worker might be required to go through login-page of NxFilter as its login session has not been created.

To address this issue as of v3.4 NxClient supports auto-switch between local network and remote network. This means that NxFilter is intelligent enough to find out if it's on local network or remote network and if it's on local network it will act as a login agent to create its login session on NxFilter and yield to the filtering of local DNS server which is NxFilter.

* Since NxClient is acting as a login agent on local network NxLogon will not start if there's NxClient already working.

* If you don't like auto-switch behavior you can add 'no_switch = 1' line info 'C:/Program Files/nxclient/conf/cfg.properties'.

Uninstalling NxClient

To prevent an accidental uninstall by user NxClient doesn't provide uninstaller for 'Add/Remove programs' in control panel. When you uninstall NxClient you need to do it manually.

- Run 'C:/Program Files/nxclient/bin/unstsvc.bat'.
- Delete 'C:/Program Files/nxclient' folder.

Silent install

Some people want to install NxClient on multiple PC using GPO or PDQ deployment. For that kind of case we have silent install option for NxClient. You can specify server-ip and login-token as install option.

For silent install,

/silent : Runs the installer in silent mode (The progress window is displayed).
/verysilent : Very silent mode. No windows are displayed.

And you can specify server-ip and login-token,

/server=192.168.0.100
/token=2P1WQ6VF

So the final form of command would be like this.

    nxclient-4.0-win.exe /silent /server=192.168.0.102 /token=2P1WQ6VF

* You can build your own MSI package using MSP wrapper from http://www.exemsi.com.

- Go index -
NxUpdate and dynamic IP update
When you have a client system using dynamic IP address and you want to associate its IP address to a specific user you can install NxUpdate on their systems. Once you tie up NxUpdate to a specific user using his/her login-token it will update the his/her associated IP address automatically.

NxUpdate has basically the same structure with NxClient. You can install it in the same way as NxClient.

* It sends START, STOP and IPUPDATE signals.

- Go index -
Application control with NxLogon and NxClient
NxFilter supports application control through its Active Directory single sign-on agent that is NxLogon and its remote user filtering agent, NxClient. You can block unwanted programs by setting up your application control rule on NxFilter GUI centrally and you can find out who tried to run the blocked programs by the log view of NxFilter GUI.

How it works

After you define your application control rule on 'Policy & Rule > Application Control' NxLogon, NxClient retrieves the application control policy periodically according to the time defined on 'Config > Setup > Misc > Agent policy update period'.

* You can adjust the policy update period for NxLogon or NxClient by changing the value for 'Agent policy update period' on 'Config > Setup'.

Supported options

1. Block by port scanning
NxLogon, NxClient detects UltraSurf and Tor processes by port scanning. This means even if your users change the process name or run them from a USB stick NxFilter can find these processes and block them.

2. Block by process name
NxLogon, NxClient supports 'block by process name'. This works based on keyword matching against process name. You can add your blocked keywords on GUI and If NxFilter finds the matching process name it will block the process.

3. Block by window title
Windows programs are supposed to have a window title. For example Skype has 'Skype' in its window title and uTorrent has 'Torrent' in its window title. You can define your keywords for matching against window title of the blocked programs.

* By default all the keywords are for partial matching but you can specify the exact matching using the square brackets. If you need to add a keyword having spaces then use double quotes.

    ex) Skype [Dropbox.exe] "Tor Browser"

* NxLogon doesn't support Unicode or multi-bytes keywords for application control.

Enable application control only for specific users

Basically the application control of NxFilter works as a global policy. However you can disable the application control for some users by checking 'disable application control' option on the 'Policy & Rule > Policy > edit' on GUI.

Logging blocked application

NxFilter is basically a dns-filter so its logging structure was designed for showing allowed/blocked domains. To accommodate the log data about blocked application NxFilter introduced these domains or rules.

- ultrasurf.port.app : UltraSurf has been blocked by port scanning.
- tor.port.app : Tor has been blocked by port scanning.
- chrome.exe.pname.app : Chrome has been blocked by its process name.
- Skype.title.app : Skype has been blocked by its window title.

Execution interval

Finding and blocking application may cause some amount of CPU load. If you don't want to cause too much load to your client PC increase 'Execution interval' on 'Policy & Rule > Application Control'.

- Go index -
Proxy filtering with NxClient
As of V2.2.2 NxFilter supports safe-search enforcing and URL keyword filtering and the other web-proxy based filtering methods. To enable these features you need to have NxClient running on user PC.

How it works

Firstly you define your proxy filtering rule on 'Policy & Rule > Proxy Filtering' and then after NxClient started on user system they will filter user web traffic by setting up itself as a local proxy server. NxClient retrieves the proxy filtering rule periodically according to the time defined on 'Config > Setup > Misc > Agent policy update period'

Supported options

1. Block HTTPS
You can block all the HTTPS traffic.

2. Block IP host
Blocking HTTP requests with IP host in URL.

3. Block other browser
NxFilter's proxy filtering is being activated through system proxy settings. Internet Explorer and Chrome are using system proxy already and many other applications are also using system proxy. But there are some applications having their own proxy setup or making direct connection to Internet. With this option enabled NxClient will block any program making direct HTTP connection to the Internet.

* Currently proxy filtering support Internet Explorer, Chrome, Firefox. The proxy setup of these browsers will be updated to use NxClient as their proxy.

* You can allow direct HTTP access to some application using 'Excluded keywords' on 'Policy & Rule > Application. Although it's for application control it is still effective against 'other browser blocking'. It works based on keyword matching against process name.

4. Blocked keyword in URL
Keyword filtering against URL.

Enable proxy filtering only for specific users

The proxy filtering of NxFilter works globally. If you need to disable it for some users check 'disable proxy filtering' option on the 'Policy & Rule > Policy > edit' on GUI.

Logging

You will get domain level log data. But you will see a detailed block reason like below.

Domain: www.google.com
Reason: Blocked by proxy, url_kw=game

- Go index -
Installing NxClient or NxUpdate on Mac OS X
When you download and uncompresss the nxclient-x.x-mac.zip file you will find these files in the uncompressed directory.

- install.sh
- nxclient
- org.nxfilter.nxclient.plist

* If you install NxUpdate just change 'nxclient' to 'nxupdate' in this tutorial.

Inside the uncompressed directory, run 'install-mac.sh' with root permission like below,

sudo ./install-mac.sh

'install.sh' will copy 'nxclient' file into '/usr/bin' and 'org.nxfilter.nxclient.plist' into '/Library/LaunchDaemons directory'. Now you need to modify the config values inside '/Library/LaunchDaemons directory/org.nxfilter.nxclient.plist'.

<string>server = 192.168.0.100</string>
<string>token = GKSYEJYG</string>

For server your NxFilter IP address and for token your user's login token. Then start it.

sudo launchctl load -w /Library/LaunchDaemons/org.nxfilter.nxclient.plist

If you want to stop it,

sudo launchctl unload -w /Library/LaunchDaemons/org.nxfilter.nxclient.plist

When you want to test connectivity or validate config values try this command.

sudo /usr/bin/nxclient "server = 192.168.0.100" "token = GKSYEJYG" -t

When you want to uninstall it,

sudo launchctl unload -w /Library/LaunchDaemons/org.nxfilter.nxclient.plist
sudo rm /usr/bin/nxclient
sudo rm /Library/LaunchDaemons/org.nxfilter.nxclient.plist

- Go index -
NxBlock for Chromebook
NxBlock is the remote filtering agent for Chromebook and Chrome browsers. It also can be used as a single sign-on agent in local network.

Installation of NxBlock

NxBlock is basically a Chrome extension. You can install it from Chrome Web Store. You can download it from the following link.

     - NxBlock download from Chrome Web Store

Filtering policy of NxBlock

NxBlock shares the policy on 'Policy & Rule > Proxy Filtering' with other proxy filtering agents. Unlike the other proxy agent it updates its policy on every 120 seconds and you can't change this value. This is for reducing the load of WebSocket communication.

Connection to NxFilter

After you install it you can see NxBlock on your extension setup panel of Chrome or 'chrome://extensions'. There is 'options' link under NxBlock icon. When you click it you will get NxBlock setup page. You need to setup these parameters for NxFilter connection.

- Sever IP : The IP address of your NxFilter.
- Login token : A login-token associated to a user on NxFilter.

Once you setup these parameters you can test the connectivity using 'Test' button. And then use 'Save' button to save and reload the new configuration.

* Every user on NxFilter has an associated login-token which can be used for identifying users with the agents of NxFilter. This login-token can be found on user edit page on NxFilter GUI.

Password protection of your setup

You can protect your NxBlock setup by having password login procedure. Once you setup a password and enable it, users will be blocked from accessing NxBlock setup page and 'chrome://extension' that is the URL of Chrome extension panel.

* You can use your admin password from NxFilter once its connection to server is established.

User identification

We use login-token and Google account to identify users. Suppose you create a user named 'chrome' and setup 10 NxBlock with the login-token associated to 'chrome'. If these users don't login to Chrome they will be appeared on NxFilter side as 'chrome' but if one of them login to Chrome using 'john1234@gmail.com' for example then he/she will be appeared as 'chrome_john1234' on NxFilter log-view.
Single sign-on in your local network

NxBlock works as a single sign-on agent on local network. We use Google account for single sign-on and we made a simple rule for this. If you login to your Chromebook or Chrome with 'john1234@gmail.com' then you need to have 'john1234' user on NxFilter. And you get the single sign-on. For the username on NxFilter side, you can create it on GUI or you can import it from Active Directory or OpenLDAP.

While the concept of single sign-on with NxBlock is easy and simple but we might have some problem from Chrome. It tries to open webpages or make DNS queries before NxBlock starts. Or it opens those webpages you were visiting before you close Chrome last time. This means it tries to access websites before NxBlock creates login-session and it leads to get redirected to login-page or DNS failures for several websites.

The solution is using a temporary username for the IP range to cover all these Chrome users. On NxFilter 'login by IP range' comes after 'login by IP session'. Suppose you create a user 'chrome' which covers 192.168.0.1 ~ 192.168.0.100. If there's a Chromebook using an IP address from the range it becomes 'chrome' on NxFilter and after NxBlock starts it appears with Google account or single sign-on username.

* Google supports Active Directory and LDAP sync with 'Google Apps Directory Sync'. If you need AD integration with your Chrombook read this, Google Apps Directory Sync

Central configuration for mass installation

When you do mass installation for NxBlock one of the problem is that you don't want to setup its connection information one by one. If you just use it as a single sign-on agent in local network it might be fine without the connection information but if you want to use it for remote filtering you must set these parameters up. At the moment NxBlock has two parameters for remote filtering. One is NxFilter IP address to connect to and the other one is login-token for authentication.

As of v1.5 we have a way for setting up these values centrally. We use a webpage and Chrome's start page function for this. Simply speaking, you write a webpage containing these config values and then make that webpage as Chrome's start page on Google admin console. Then everytime your users start their Chrome they will setup themselves with the values.

When you write the webpage you use meta tag like below,

<meta name='nxblock' content='192.168.0.100:HW00IYKW:1'>

We have 3 parameters separated by colons. The first one is NxFilter's IP address and the second one is the login-token and the last one is about locking or unlocking Chrome's extension setup page. When you lock it you still can use your NxFilter admin password to access Chrome's extension setup page.

You can add this meta tag into your existing webpage if you have a start page for your Chrome users already or you can write a new one and upload it into some webserver or possibily NxFilter's built-in webserver.

On Google admin side,

1. From the main dashboard, go to Device Management > Chrome > User Settings.

2. Select the organizational unit to which you want the settings to apply.

3. Find 'Pages to Load on Startup'.

4. Enter the URL for the web page containing NxBlock configuration meta tag.

5. Click the 'Save Changes' button.

- Go index -
What is NxCloud?
NxCloud is a fully rebrandable multi-tenancy cloud based webfilter software. It is based on NxFilter and inherited the most of the features of NxFilter. Simply speaking, you can build your own cloud filtering service like OpenDNS.

These are the features only available on NxCloud.

Multi-level admin

If you want to build your own cloud service one of the essential factors would be being able to create accounts for your customers and the customers need to be able to setup their own policy on their own GUI.

On NxCloud there are 3 kinds of users.

    Admin > Operator > User

'Admin' is actually the admin of NxCloud. It has almost same GUI as in the freeware version but being an admin you can create operator accounts. These operator accounts are for your customers and it's something like a sub-admin on NxCloud. They can create and manage their own users and policies.

Creating an operator

To create an operator you need to login to the GUI with admin account. On 'Config > operator' you can create an operator. When you create an operator NxFilter creates a default user and a default policy for the operator with the same name. After you create an operator you can change the number of users and policies the operator can create. This means you can have several levels on your service based on the permission for an operator.

Operator GUI

On NxCloud each operator have their own GUI. If you login to the GUI with an operator account you will be on the operator mode GUI. It's a bit restrictive compared to admin GUI as you only can setup operator specific parameters.

Operator and user

Operators can create their own users and apply different policies to each user. Users can be authenticated based on IP addresses or password or using NxClient. If you are on Active Directory environment you can use NxLogon for single sign-on with your Active Directory.

Operator specific dashboard and report

Dashboard and report of NxFilter is still available on operator GUI.

Operator specific free-time

Each operator can define their own free-time and they can setup a work-time policy and a free-time policy for their user.

Operator specific whitelist and blacklist

You can add operator specific whitelist/blacklist based on domain name. But you still have the global whitelist/blacklist for admin. So you can have more flexibility to deal with these whitelist and blacklist as an admin.

Operator specific alert-email

NxCloud sends alert email about the blocking incidents to each operator. Operators can setup their email addresses to receive the email and define alert period on 'Config > config'.

* You need to setup global alert email setup first to send the operator specific alert email. You can setup this global alert email on 'Config > alert' of admin GUI.

Operator specific block-page

Each operator can have their own block-page. If there's no block-page defined by operator NxCloud shows the default block-page by admin.

Authentication over cloud

NxClient still works against the cloud version. This means you can differentiate users behind their router and you can apply different policies on different users.

Dynamic IP updater

Many of your clients will be using the service from dynamic IP addresses. So one of the essential factors for your cloud based web-filter service would be having a dynamic IP updater. You can use NxClient for this purpose.

Dynamic DNS association

We provide NxUpdate for associating a dynamic IP address to a user. But some of your users may already have a dynamic IP updater for their dynamic DNS service. They don't want to install one more thing on their system. For that reason, we support the association between a user and a dynamic domain. You can add a domain instead of an IP address as an associated IP/domain on user edit page.

Rebranding or customization of GUI

Its GUI layer is designed for easy customization. The GUI layer is separated from its core part. You just need to modify all the JSP pages in '/nxfilter/webapps' directory. These JSP files have a naming rule corresponding to the GUI menu structure. So it's easy to find which file you need to modify.

- Go index -
Pricing plan for NxCloud
The pricing is 3 USD per-user, per-year including Komodia cloud blacklist option. We allow you to have your own customized GUI. We also allow you to customize our agents but some people find it difficult so we can change the name and icon and text in the setup process of NxUpdate and NxClient when you buy more than 2,000 user license. We accept the payment through PayPal.

* When you increase NxCloud license user number you can add it by 500 user block. For example, if you need to add 500 user license 6 months after you start your business then its price would be for the remaining 6 months and then next year you renew all the license at once.

- Go index -
Install NxCloud
NxCloud is designed for multi-tenancy cloud based webfilter service. It has almost every features of NxFilter. You can install and run NxCloud in the same way as you do with NxFilter.

But unlike NxFilter after you install it you can't use it as your DNS server right away. This is because NxCloud is a multi-tenacy program for commercial service. You're not supposed to use it for your internal network. Your clients use it for their network. So you need to create an account for your client first.

On NxCloud there are 3 kinds of users.

Admin > Operator > User

'Admin' is you and an operator is your customer and a user is the user in your customer's network. An admin manages operator accounts and an operator manages the end users and policies. So you need to create an operator first. To create an operator login to NxCloud GUI as admin and then go to 'Operator' menu. You can create an operator there.

When you create an operator there will be a default user and default policy for the operator with the same name as the operator. And the default password for the operator is also the name of the operator. Once you create an operator you can login with the operator account to setup a user for testing.

* You need to associate your IP address to the default user of your first operator to test it.

- Go index -
Differences from running NxFilter
1. Authentication enabled always
You don't want to make your service available to everybody for free. You want to service it to your clients only. so there's authentication enabled by default. Another reason for enabling authentication is that your NxCloud can be a target of DDOS attack. So you need to service it only for the known clients.

2. Login redirection disabled at default
You still can use password based login with NxCloud but if you use that on public network you can be a target of DDOS attack. You'd better disable it on public network. When you disable it NxCloud doesn't respond to any unknown user or client. So you'd need to use login redirection or login-page only when you service NxCloud in a private network or on VPN.

3. Magic password for accessing operator GUI
As an administrator of NxCloud sometimes you would need to access operator GUI for technical support purpose. For that reason NxCloud has one more passowrd for admin. It is called 'magic password'. With this password you can access any operator's GUI. The default magic password is 'magic1023' and you can change the password on 'Config > Admin'.

- Go index -
Business account and home account
When you build a cloud based filtering service one of the problems you have would be finding out the exact number of users behind a router. It may be possible when there are some agents installed and running behind router and NxCloud supports several agents for that. However some of your customers don't need to differentiate users and they just want to have one global policy for every users. It means you don't know how many users they have.

To address this issue we limit the request count for a user. Currently one user can make 3,000 requests a day. This is more than enough considering a user makes under 1,000 requests a day according to our statistics so far. However we may have another issue from this request count limit approach. If you have a customer using your service in their home they probably have several Internet accessing devices and have several family members but not wanting to pay for multiple users. In that case this 3,000 daily request limit is too small for them.

To solve this problem we introduced the operator type concept. There are 2 kinds of operators on NxCloud. One is 'Business' and the other is 'Home'. Business type operators are nothing special. You can create as many users as you want for them and each of the users has 3,000 request limit. But if it is a Home type operator its first user has 7,000 extra request count and that makes 10,000 daily request count limit for the first user. If it has 2nd user its request count limit becomes 13,000 and that would be more than enough for most home users.

* For home type operator you can create up to 5 users.

* To reduce the number of requests set 'Max client cache TTL' to 0 on 'Config > Setup'.

- Go index -
Building your own billing system for NxCloud
Building your own billing system for NxCloud When you service NxCloud commercially you want to have an automated billing system or you want to create and manage your client account on your side. As all the GUI layer exposed as JSP pages it's not that difficult for you to build your own builling system with NxCloud.

To build your own billing system you need to be able to create, edit an operator which is your client account on your own webpages. Suppose if you need to create an operator with these properties.

- Name : triton
- Password : triton1234
- Email : tmail0487@yahoo.com
- Max user : 3
- Max user IP : 3
- Max policy : 3
- Max whitelist : 20
- Max free-time : 10

The JSP code would look like below.

<%
OperatorData data = new OperatorData();
data.name = ”triton”;
data.passwd = ”triton1234”;
data.email = ”tmail0487@yahoo.com”;
data.max_user = 3;
data.max_user_ip = 3;
data.max_policy = 3;
data.max_whitelist = 20;
data.max_free_time = 10;

OperatorDao dao = new OperatorDao();
dao.insert(data);
%>

If you need to update the properties of an operator.

<%
OperatorDao dao = new OperatorDao();

OperatorData data = dao.select_one_by_name(”triton”);
data.max_user = 5;
data.max_user_ip = 5;
data.max_policy = 5;
dao.update(data);
%>

If you need to suspend an operator.

<%
OperatorDao dao = new OperatorDao();

OperatorData data = dao.select_one_by_name(”triton”);
data.stop_flag = true;
dao.update(data);
%>

* There is a separated section for GUI customization for NxFilter on this tutorial and we also provide Javadoc for building your own custom GUI.

- Go index -
Directory structure and naming rule
The GUI layer of NxFilter was designed for easy customization. It is completely separated from its core part. And it has a naming convention corresponding to its menu structure so that you can find the file you need to modify easily. For example if you want to modify 'Policy & Rule > Free Time' on NxFilter menu the file you need to edit is '/nxfilter/webapps/policy,free_time.jsp'.

* In NxCloud's case it has operator specific menu. If a JSP file is for operator specific menu then it has 'zop' prefix.

    ex) zop,policy,free_time.jsp

Structure of web application directory

We put all the JSP pages into '/nxfilter/webapps' and we don't use any subdirectory for keeping JSP pages. This is for the simplicity and easy understanding. So everything you need to modify is in '/nxfilter/webapps' directory. It has the following structure.

/nxfilter/webapps
- error
- example
- img
- include
- lib
- WEB-INF

In 'webapps/error' directory we have error pages for HTTP error codesIf you want to have the error pages for other error codes you can define it on '/webapps/WEB-INF/web.xml'.

* We use 400 error for special purpose. You shouldn't define any error page for 400 error.

In 'webapps/example' directory we have some example JSP pages for custom login module.

In 'webapps/img' we keep image files for webpages.

In 'webapps/include' we have common JSP files to be included into the other JSP files. These are for library functions and navigation menus and initialization code for JSP pages.

* '/include/lib.jsp' is a common library file for all JSP files. It has some utility functions for web development and it executes initialization code for JSP pages and does authentication checking as well.

* We don't include '/include/lib.jsp' directly. It gets included when we include '/include/top.jsp'.

In 'webapps/lib' we have CSS and javascript files.

We have 'WEB-INF' directory as we use an embedded Tomcat as our webserver.

Separating your customized GUI into another directory

When you customize NxFilter's GUI it is not a good idea to modify original files directly. You'd better keep it for future reference and create another directory under the installaion directory of NxFilter and copy all the files inside '/nxfilter/webapps' into the new directory and then modify these copied files. For this, NxFilter supports 'www_dir' option on '/nxfilter/conf/cfg.properties' file.

So if you have your own custom GUI in '/nxfilter/myweb' directory. You need to add this line into your cfg.properties file.

    www_dir = myweb

Then restart your NxFilter.

- Go index -
Using Dao and Data classes
On typical web programming, dealing with DB is almost everything. We are using 'data access object' and 'data object' concept for manipulating data and separating GUI layer.
Common methods for data access object

We have some common methods for most data access object classes. For example on 'policy,policy.jsp' file we use PolicyDao and PolicyData class for manipulating policy. PolicyDao has these methods.

public int select_count() : The number of policies.
public List select_list() : Fetching policies as a list.
public PolicyData select_one(int id) : Fetching one policy by ID column.
public boolean insert(PolicyData data) : Insert a new policy.
public boolean update(PolicyData data) : Update a new policy.
public boolean delete(int id) : Delete a policy by ID column.

Every policy data has its own unique ID which is a number and we use this ID for finding, updating a policy data.

Insert, delete, update, select data

If we want to modify 'whitelist,domain.jsp' we have to use 'WhitelistDomainDao' and 'WhitelistData' classes.

To insert a new data,

<%
WhitelistDomainDao dao = new WhitelistDomainDao();

WhitelistData data = new WhitelistData();
data.domain = "*.nxfilter.org";
data.bypass_auth = true;
data.bypass_filter = true;

dao.insert(data);
%>

To delete a data when its ID is 12,

<%
WhitelistDomainDao dao = new WhitelistDomainDao();
dao.delete(12);
%>

To select a data when its ID is 12,

<%
WhitelistDomainDao dao = new WhitelistDomainDao();
WhitelistData data = dao.select_one(12);
%>

And to update the selected data,

<%
data.bypass_filter = false;
dao.update(data);
%>

Lastly, to list data.

<%
WhitelistDomainDao dao = new WhitelistDomainDao();
List data_list = dao.select_list();
for(WhitelistData data : data_list){
    out.println(data.domain + "<br>");
}
%>

Accessing data field

Many Java developers are using 'get' and 'set' accessors for encapsulation and for having some additional data processing like validation but we use public field directly in most cases. For example you get an instance of UserData and uses its 'name' property like below,

<%
UserData data = new UserDao().select_one(1);
out.println(data.name)
%>

But there are some Data classes having methods starting with 'get_'. These methods are mostly about formatting. We have 'ctime' property for RequestData which is being used on 'Logging > Request'. If you use it directly you get '201507081415' but when you use its get_ctime() method you get '07/08 14:14'.

- Go index -
Javadoc for Dao and Data classes
We have Javadoc for Dao and Data packages.

- Go index -
Clustering with NxFilter
NxFilter has built-in clustering for load balancing and fail-safe. Once you have a master node you can add up to 4 slave nodes to your cluster. All the slave nodes in your cluster sharing the setup from the master node. So you can control everything on your master node.

Creating a cluster

To create a cluster firstly you need to make a master node. On 'Config > Cluster' you can make your NxFilter to be a master node. And then you can add the other NxFilter installation as the slave nodes to your mater node. You need to restart NxFilter after change cluster setup.

Starting clustered NxFilter

When you start NxFilter cluster you need to start the master node first and then you can start the slave nodes. This is because your slave nodes need to download the initial setup from the master node when they start.

* If your slave node doesn't work properly there might be a connectivity issue between your cluster nodes. TCP 19003, 19004 ports need to be opened for you cluster nodes to communicate with each other.

Load balancing and fail-safe

One good thing about dns-filter is that there's already a method of load balancing and fail-safe existing. Make your clustered NxFilter to be the primary and secondary DNS servers for your client PC. Then you get the load balancing and fail-safe.

* If you want to have load balancing and fail-safe for your block-page and login-page or policy update for your NxLogon and NxClient agents you need to set multiple block redirection IP separated by commans on 'Config > Setup'.

When a cluster node down

When a slave node down the other slave nodes and master node will not be affected. The rest of the nodes will be working normally. But when your master node down you lose filtering but DNS lookup still working. When the master node restored your slave nodes re-establish the connection to the master node automatically.

If you setup the alert email on 'Config > Setup' you will receive an email for cluster node down.

Access control for slave nodes

If you add all your slave node IP addresses into 'Config > Cluster' on the GUI of your master node any unregistered IP address will be blocked from the master node.

Using a different set of resolving DNS servers for a slave node

You might want to have a fail-safe measure for the connection to your resolving DNS servers by having a different set up resolving DNS servers for your slave node. But the problem is that in a cluster every node share a master node's configuration. So you can't do it using GUI. However you can use '/nxfilter/conf/cfg.properties' for that purpose. There's 'upstream_dns' param you can add into the file. For example if you want to use '192.168.0.100' and '192.168.0.101' as your upstream servers you can add this line into 'cfg.properties' file.

    upstream_dns = 192.168.0.100,192.168.0.101

Monitoring slave node state

You can view connection state from your slave nodes on 'Config > Cluster'. Once you setup your cluster then your slave nodes will appear with the last contact time on the page. It is also showing request, block, user, client-ip count information. These counter information will be set to 0 on midnight or when you restart NxFilter as they are being stored in memory.

Session sharing between cluster nodes

When you build a cluster one of the problems we need to solve is sharing data between nodes. For NxFilter we use TCP ports for sharing data. The one of the data we share is the login session so that you don't need to login twice to master and slave nodes. And we share quota-time and bandwidth consumption data as well. But this could be a reason for performance degrading when you have busy servers as it increase the amount of communication between nodes.

If you don't want to share these data you just need to disable authentication and not to use quota-time and bandwidth control. But while the other two are doable options, not sharing login session and showing users NxFilter's login-page more than twice is not an option you want to go with. So we have 'no_share_session' option which diables sharing login session even if you have authentication enabled.

It is possible as this login session sharing is only for login-page. If you don't use login-page or login by password you are not going to have any problem. NxLogon and NxMapper, NxClient can deal with multiple servers. And IP based authentication always works fine regardless of session sharing. To disable session sharing add this line into '/nxfilter/conf/cfg.propertie'.

    no_share_session = 1

* You need to setup 'no_share_session' option on all the slave nodes.

- Go index -
Bandwidth control with NxFilter
NxFilter supports per-user based bandwidth control. The idea is simple. Using NetFlow data NxFilter associates the traffic data to its user login session and if there's a user consumed bandwidth over the limit NxFilter blocks all the DNS requests from the user.

One good thing is that it is not just about HTTP traffic. Since NxFilter uses NetFlow data you can monitor and block other protocols including HTTP, FTP, torrent, streaming , IM, Skype and any other protocols working on TCP/UDP.

To enable bandwidth control you need to have a router or firewall supporting NetFlow version 5 in your network. If you have NetFlow v5 supporting router setup it sending NetFlow data to NxFilter. And then run the built-in NetFlow collector of NxFilter. You can run the collector on 'Config > Setup > NetFlow'. Then you can setup per-policy bandwidth limit.

There are several rules for NxFilter importing NetFlow data. Firstly the source or destination IP addresses of a NetFlow data need to be associated to one of the IP addresses of logged-in users of NxFilter. Secondly NxFilter ignores internal traffic. This means one of the source or destination IP addresses needs to be a public IP. This is because you are only interested in inboud or outbound traffic from/to the Internet. And lastly NxFilter keeps only TCP/UDP data.

* When you have a cluster there might be up to 1 minute delay for the update of 'blocked user list by bandwidth'. This is for preventing too much frequent communication between the master and slavess.

* Currently NxFilter supports NetFlow v5 only.

- Go index -
Detecting and preventing malware/botnet activity with NxFilter
One of the features of NxFilter is being able to detect and block malware/botnet activity by analyzing DNS packet. In reality malwares and botnets are another form of network clients or server programs. This means they are also heavily relying on DNS protocol to find their master servers or peers to communicate with each other.

For example if you have a spambot in your network the spambot will make a lot of DNS queries for MX records for target domains to send emails. But normally your client PC doesn't need to make MX queries unless they have an email server running on it.

Another example would be the botnets using the 'TXT' record or other DNS records as their communication tool. These are the real world examples of malwares using DNS protocol as their communication tool.

ex1) Trojan.Spachanel was using SPF record.
ex2) W32.Morto was using TXT record.

The other method we can think of would be detecting abnormal length of domain. When we tested top 100,000 domains from www.alexa.com all the domains except 142 domains were shorter than 30 characters. But there are abnormal domains trying to look like some URL of target domain. This is an example of www.phishtank.com which is trying to look like a webpage of www.ebay.co.uk but actually it is a phishing domain.

ex1) cgi.ebay.co.uk-item-css.ebay-motors.session.id-sj3mzbasf3k12z581668115.login-wpadmin-sw.buyitnow.sign-in.secure-process657943sddh53zix34235hj65rj.xml.config-page.overview.buyer-protection-jsp.wpcs.spiridus-magic.org

So detecting botnet/malware by analyzing DNS packet would be one of the effective technique we can think of. NxFilter provides these blocking options on its policy setup.

- Max domain length
- Block covert channel
- Block mailer worm
- Block DNS rebinding
- Allow 'A' record only

But you can say that the most effective way of preventing malware/botnet in your network would be allowing only 'A' record query from your client PC. In most cases your client PC doesn't need to make a DNS query for any records except 'A', 'AAAA', 'PTR' , 'CNAME'. If you have some email servers or the other network servers in your network then you can apply a different policy for them or bypass them from NxFilter.

- Go index -
Removing embedded adverts in webpages
As of v2.2.7 we added Javascript code hiding embedded block-page into the default block-page. If you updated it from an older version use 'restore-default' button to enable it.

There are webpages having embedded adverts from other domains. One of the problems for blocking these adverts with NxFilter would be having a mangled webpage as the result of blocking. Your block-page replaces the embedded adverts.

To avoid of having this kind of problem there are 2 ways of removing embedded adverts with NxFilter. One is using a special category in 'Category > Custom' which is called 'ad-remove'. If you add some domain into this category and you block the category somewhere NxFilter blocks the category with blank block-page.

The other method is to block it using the 'Ad-remove' option on a policy. With this option NxFilter blocks 'adv' category of Shallalist. If you want to use this option you need to import Shallalist first.

* After you add a domain into 'ad-remove' category you need to block it on whitelist or policy otherwise it will not be blocked.

- Go index -
Syslog exportation
NxFilter provides syslog exportation function. The exported data is a character string that you can split by '|'. For example if you have the following syslog data.

NXFILTER|2013-01-28 10:53:23|Y|www.bbc.co.uk|pwuser|192.168.0.101|admin|news|Blocked by admin|33

It can be parsed into these values.

- Prefix : NXFILTER
- Date : '2013-01-28 10:53:23'
- Block yes/no : Y
- Domain : www.bbc.co.uk
- User : pwuser
- Client IP : 192.168.0.101
- Policy : admin
- Category : news
- Blocked reason : 'Blocked by admin'
- DNS query type : 33

- Go index -
Performance tuning guide
NxFilter was designed to handle more than several thousands users. Currently, 2014-06-21 the biggest site has more than 7,000 users and they are using 2 NxFilter clustered. I believe you don't need to have high-end machine to see this kind of performance. Just ordinary PC hardware will do the job.

However this does not mean that you get the best performance without proper system requirements. If you don't have enough resource available while having too many users you will end up having slow response from NxFilter. To get the best performance you can adjust several factors for NxFilter.

Memory size

At default NxFilter uses up to 512M of RAM. This is enough for most users. But if you allocate more memory to NxFilter you can expect better performance. In NxFilter startup scripts, '/nxfilter/bin/startup.bat' you have the following line.

    java -Djava.net.preferIPv4Stack=true -Xmx512m -cp "%NX_HOME%"\nxd.jar;"%NX_HOME%"\lib\*; nxd.Main

If you want to increase it to 1G then change '-Xmx512m' to '-Xmx1024m'. When you change this value you also need to think about the redundant memory for other programs on your system. So if you allocate 1G of memory to NxFilter you'd better have more than 1.5G of memory on your system at least.

Disk space and reducing the amount of log data

NxFilter has some fancy reporting feature. You can view all the logging data and daily, weekly report and per-user report. However this kind of reporting consumes a lot of disk space. When you have bigger size of reporting data your system might have some performance degrading.

If you have more than several hundreds users you'd better have at least 10G of disk space for traffic DB. Another option is to reduce the amount of data. To reduce the amount of traffic data you can adjust the value for 'Log retention days' on 'Config > Setup'. The default value is '30'. It means NxFilter keeps its log data for 30 days.

The other way of reducing the amount of traffic data would be using whitelist with 'Bypass logging' option. For example you can bypass 'e3191.dscc.akamaiedge.net' or 'us-courier.push-apple.com.akadns.net' domains for logging when you have too many requests for these domains from Apple devices.

Use client cache for DNS response

NxFilter manipulates DNS cache TTL to clear out the client DNS as soon as possible to avoid of having confusion from the client side DNS cache but basically this is not a critically needed function and it increases the number of DNS queries from your client.

You can set 'Max client cache TTL' in 'Config > Setup' to '0' to turn off the function. When you turn it off your client PC will not make DNS query against the domain already existing in its cache and it will reduce the load for NxFilter significantly. When you have more than 1,000 users we recommend you to turn off this function.

Increase the number of request handlers

NxFilter is a multi-threaded program. It has worker threads handling client DNS requests. The default number of request handlers is 8 and it's enough for most cases. But if you think your NxFilter responding slowly you can try to increase it. To increase it to 16 add the following line into '/nxfilter/conf/cfg.properties' and restart NxFilter.

    rh_num = 16

Using local recursive DNS server

One of the possible cause of performance degrading for NxFilter would be its lacking of recursive query. This is not the case when you have just several hundreds users as NxFilter has its own caching. But if you have several thousands users or if you service it over cloud this could be an issue. So we added local recursive DNS option.

However this doesn't mean that NxFilter does recursive DNS query by itself. Rather you can install a recursive DNS server into same server with NxFilter and make NxFilter uses it as its upstream DNS server. So now we have 'local_resolver_port' option on cfg.properties file. If you install something like MaraDNS's Deadwood recursive DNS server and set it to use 10053 port and listening '127.0.0.1' then you need to add this line into cfg.properties file.

    local_resolver_port = 10053

And then restart NxFilter.

Disable data sharing between cluster nodes

When you use a cluster NxFilter makes a communication channel between cluster nodes for data sharing. This could be a performance degrading factor when you have busy servers. To reduce the amount of the communication for data sharing read Clustering with NxFilter section on this tutorial.

- Go index -
Differences between agents
NxFilter provides several agents. Some are for single sign-on with Active Directory. Some are for remote user filtering and dynamic IP update. Some of them can be used for application control and proxy filtering.

1. NxLogon
Sigle sign-on agent for Active Directory. You can launch it from a logon script on GPO. It supports application control.

2. NxMapper
Another way of Active Directory integration or single sign-on. Unlike NxLogon you install it on a domain controller as a Windows service. It detects user logon on Active Directory and creates login session for the user on NxFilter.

3. NxClient
Remote user filtering agent of NxFilter. When you have a mobile worker or home worker you can install NxClient on their laptop. NxClient will be running as a Windows service and filtering user Internet activity. It supports application control and proxy filtering.

4. NxUpdate
Dynamic IP updater for NxFilter.

5. NxBlock
Remote filtering agent and single sign-on agent for Chromebook or Chrome browser.

- Go index -
User contributed documents and scripts
- Chad Coccioniti's script to auto-update URLBlacklist on Windows.
     How to auto-update URLBlacklist

- Tutorial for how to install NxFilter on Ubuntu by Carl Miller.
     Install NxFilter on Ubuntu for beginners

- Rob Asher's NxFilter start/stop script for Linux.
Rob Asher sent me a script to start and stop NxFilter with Linux system. I attached Rob's script and you just need to copy it into '/etc/init' directory on your Linux.

Rob said:
I threw together a little upstart script to manager nxfilter a bit easier on linux systems. I'm using it on CentOS 6.5 but it should work on any linux that uses upstart scripts like RHEL/CentOS/Ubuntu and the file paths match the script. Copy the attached file to /etc/init/ and nxfilter will startup and shutdown with the system plus you can control it with commands like "start nxfilter" or "stop nxfilter" or "status nxfilter". download-script

- Rob Asher wrote a single sign-on script from Linux and Mac clients using NxFilter's login API for custom login script.
     NxFilter + OS X and linux login script

- Mark Page wrote some excellent documents and scripts for NxFilter. It's especially useful if you are an advanced user.
    Of Little Consequence

- Rob Asher's JSP page for restarting NxFilter from GUI.
     View posting

- Stewart Sentanoe's 'Simple way to change blocked page GUI'.
     View posting

- Go index -
FAQ
These are frequently asked questions about NxFilter.

I can bypass NxFilter by accessing websites using IP address.

There are people saying that DNS filtering is useless as they can access websites using IP address. This is very naive thought and simply not true. In today's Internet environment most websites are running on the virtual host. This means there are multiple websites on one IP address. You can't access these websites without domain. And the other reason is that there are many URLs in a webpage. This is especially true when it comes to big portal sites. Those URLs are based on DNS as well. So if there's no DNS resolving the webpage you're trying to view will not be loaded properly in most cases.

* NxFilter can block IP host in URL with its local proxy agents.

- Go index -

It doesn't get blocked/unblocked right away.

This is most likely from the DNS cache on your system. There are 2 kinds of DNS cache on your PC. One is from your browser and the other from Windows OS. Before the cache expires your policy change for blocking/unblocking will not be working. Both caches expire eventually but you might want to clear out manually. If it is the browser cache you can clear it out by restarting your browser.

If you want clear out your Windows DNS cache use the following command in your command-line console.

ipconfig /flushdns

Normally DNS cache from Windows expires in a day at the maximum. Of course it depends on TTL from DNS record but I didn't see it bigger than 86,400 seconds(1 day) usually. About the browser cache it may take several minutes to get expired. But it will get expired and blocked eventually. So in practice this is not a problem as you don't need to block/unblock a site many times a day.

- Go index -

I still get blocked after I logged in.

It is from your browser cache. The fastest solution to this problem is restarting your browser. Or you can wait until the browser cache expired. But it may take several minutes.

There is a workaround for this problem. You can use 'Login domain' in 'Config > Setup'. If a user type the login domain the user gets the login-page right away. You can make this domain to be the start page of the browser.

* If you use single sign-on with Active Directory you can avoid of having this kind of problem.

- Go index -

How do I force users being filtered by NxFilter?

If you have a firewall in your network it's simple. You just need to block all outgoing 53 UDP/TCP traffic except coming from NxFilter. And then you use DHCP to setup NxFilter to be the DNS server for your network. Now NxFilter became the only DNS server your users can use and their DNS setup to point NxFilter being done automatically.

- Go index -

How NxFilter decide which policy to apply for a user?

You can assign a policy to a user directly. If the user belongs to a group then group-policy overrides user-policy. To this it's simple but the tricky thing is that when import users from Active Directory. In NxFilter you can not assign a user to multiple groups but in Active Directory you can.

To solve this problem I introduced 'priority points' concept. If there are multiple groups and if they have several different policies the policy having highest priority-points will be applied. You can set priority-points by editing policy. You can view which policy being applied with 'test' button in 'User & Group > User'.

- Go index -

What's the quickest way of blocking 'facebook.com'?

Add '*.facebook.com' into 'Whitelist > Domain' and check 'admin_block' option. This becomes a global blacklist applied to everyone.

- Go index -

I want to block 'facebook.com' only for students.

Create a user or a group for your students and create a policy for the user or group. And then create a custom category to add '*.facebook.com' into. Then in the policy block the category and assign the policy to the user or the group. If you use Shallalist you can just block 'socialnet' category without creating your custom category.

* This is a user specific blocking you need to enable authentication in 'Config > Setup' first.

- Go index -

I want to allow sales dept to use the Internet freely in lunch time.

You can create a user or a group for your sale dept and define a free-time in 'Policy & Rule > Free Time' then you can assign a free-time policy which is lenient to the user or group.

- Go index -

How do I change webserver port?

You can change HTTP/HTTPS listening ports on NxFilter. However when you change HTTP port you will lose block-redirection. It's because when NxFilter redirects user browser there needs to be something waiting for the browser on 80 port. And this is the block-page from NxFilter.

To avoid of having confusion for most users I didn't make it on GUI. You need to modify 2 parameters '/nxfilter/conf/cfg.properties' file.

http_port = 80
https_port = 443

After you change the ports you need to restart NxFilter.

- Go index -

How do I reset admin password?

NxFilter provides '/nxfilter/bin/reset_pw.bat' util scripts to reset the password. Once you run these scripts admin name and password to GUI will be reset to 'admin'. When you execute these scripts NxFilter must be running.

* There is '/nxfilter/bin/reset_acl.bat' to reset access restriction to GUI as well.

- Go index -

Can I bind NxFilter to a specific IP address?

You might want to bind NxFilter to a specific IP address to avoid of having port collision problem. You can bind NxFilter to a specific IP address using 'listen_ip' parameter in '/nxfilter/conf/cfg.properties' file. If you set it to '0.0.0.0' NxFilter will listen to all the IP addresses of the system but if you set it to a specific IP address NxFilter will listen to the specified IP address.

* Even if you bind NxFilter to a specific IP address you can not run multiple NxFilter on the same machine. You still can have port collision problem as there are other ports being used by NxFilter internally.

- Go index -

How do I bypass my local domain?

As of v1.6.1 you can bypass queries for your local domain. This is different from whitelist as it gets no filtering and no logging. You can set your local DNS server and local domains and then if there are queries for your local domains NxFilter forwards the queries to the local DNS server and bypass the filtering and logging.

There are 2 parameters you can add into '/nxfilter/conf/cfg.properties' file. One is 'local_dns' and the other is 'local_domain'. For 'local_dns' the IP of your local DNS server and for 'local_domain' your local domains which you want to bypass from NxFilter. You can have multiple domains separated by commas.

local_dns = 192.168.0.101
local_domain = rainbowx.local, mydomain.local, yourdomain.com

- Go index -

How do I update Shallalist nightly?

If you want to update Shallalist on midnight you can write your own script and add it into crontab if it's on Unix.

#!/bin/sh /nxfilter/bin/shudown.sh
/nxfilter/bin/update_sh.sh
/nxfilter/bin/startup.sh -d

If you are on Windows use 'net start NxFilter' and 'net stop NxFilter' commands. It would look like below.

net stop NxFilter
c:/nxfilter/bin/update_sh.bat
net start NxFilter

- Go index -

Too many requests for some domains.

One of NxFilter users reported that he gets too many requests for these domains.

e3191.dscc.akamaiedge.net
us-courier.push-apple.com.akadns.net

According to him he has almost 1,200 users in K-12 school environment and they are using only Mac. These domains being used for CDNs for Apple and he gets about 100k of requests for these domains a day. The problem of this kind of situation would be that you can't view the other domains on the log-view or report and as a result it would be difficult for you to understand what's going on in your network. And it will also slow down your NxFilter with the massive amount of data. To avoid of having this kind of problem what you can do is using whitelist with 'bypass_logging' option. Once you bypass these domains or the domains being used for some special purpose you will have lesser load for NxFilter and lesser amount of log data.

* If they are important domains for your network we recommend you to set 'bypass_authentication', 'bypass_filter' options as well.

- Go index -

Can I use exact matching for log search?

You can use square brackets for exact matching on log search.

    ex) [john], [192.168.0.1]

- Go index -

I want to disable login redirection.

There's no config option for that. However you can achieve what you want by emptying your login-page or just having the message like 'We don't have a login-page for you!' instead of a login form.

- Go index -

Why do I need to re-login after lunch break?

Your login session has been expired. If there is no activity(DNS query) from your PC for certain amount of time then your login session expires. You can increase IP session TTL in 'Config > Setup'.

* If you use single sign-on with Active Directory you can avoid of having this problem.

- Go index -

How do I apply my own SSL certificate?

We use an embedded Tomcat 7.x as the built-in webserver for NxFilter. If you want to apply your own SSL certificate with Tomcat there are 2 parameters you need to set in Tomcat config file. One is 'keystoreFile' and the other one is 'keystorePass'. As we don't have the separated config file for Tomcat we use '/conf/cfg.properties' file to set these parameters.

keystore_file = conf/myown.keystore
keystore_pass = 123456

* About how to build keystore file read Tomcat manual.

- Go index -

How do I enable debug mode?

When there's something wrong with NxFilter the first thing you need to do is finding out what is going on exactly with its log data. It keeps system log data inside '/nxfilter/log' directory. If you need more detailed log data you can enable debug mode on '/nxfilter/conf/log4j.properties'. Change 'INFO' to 'DEBUG' inside the file and restart NxFilter.

- Go index -

How do I hide SSL warning?

When a browser being redirected on HTTPS it warns users that they are being redirected. This is for preventing so called 'Man in the middle' attack. That's why you get those SSL warning pages instead of NxFilter block-page on HTTPS. So your browser is just doing its job and we don't want to interfere that. However we know that there are users wanting to hide these warning pages for some reason. We still can't show block-page on HTTPS but you can hide it by changing SSL port of NxFilter. If you use a non standard SSL port which is not 443 then your browsers get to nowhere when they are redirected and they will show 'connection error' message.

To change SSL port for NxFilter modify this line on /conf/cfg.properties file.

https_port = 443

- Go index -

I don't see any username on log-view.

The first thing you need to check would be 'Enable authentication' option on 'Config > Setup'. Many people forgot that they need to enable authentication before implementing any authentication method. After that if it's about Active Directory integration you would need to import users and groups from your Active Directory so that NxFilter can recognize your AD users.

- Go index -

How do I bypass logging completely?

For internal purpose the minimum log retention period you can set is 3 days. But you can bypass logging completely by setting 'syslog_only' option /conf/cfg.properties file. If you set this option without having Syslog exportation setup then NxFilter bypasses logging and not sending Syslog data as it doesn't know where to send it.

To enable 'syslog_only' option add the following line on /conf/cfg.properties file.

syslog_only = 1

* You still get the counting data but the actual logging data will not be stored into your traffic DB.

- Go index -