Black Hat USA 2012 – Emulex FastStack Sniffer10G Product Demo Shows 2x Suricata Performance Gain

scottcschweitzer Capture, networking, Security

This article was originally published in July of 2012 at 10GbE.net. 

At Black Hat USA this week at Caesars Palace in Las Vegas, Emulex & Myricom will be demonstrating Sniffer10G as it greatly improves Suricata performance by a factor of two. Suricata is a high-performance open source application for creating an intrusion prevention (IPS) or intrusion detection system (IDS). Snort is the leader in this space, but Suricata is the popular new application looking to capture early and mass adoption. While Snort was designed as a single threaded application, Suricata was built from the ground up to support multi-core servers. Sniffer10G takes this one step further, leveraging the processor found on Myricom and Emulex NX adapters to provide multiple capture buffers that can then be pinned to specific CPU cores.

This blog entry specifically outlines how to accomplish a 2X performance improvement for Suricata by using to Sniffer10G. First, you will need to purchase a high-performance dual port 10GbE adapter with Sniffer10G: Myricom dual port 10GbE (P/N 10G-PCIE2-8C2-2S+SNF2) or an Emulex NX adapter (P/N OCe12102-DX-SNF2). One of these cards is required to run Sniffer10G which is also required for performance gain. Both adapters can be purchased for $1,000.00.  This price includes Sniffer10G software. 
For the sake of this posting, we will assume you know how to install the server, the network adapter, and the OS. Prior to deployment, and for testing purposes only we will use a single DA cable to connect together both 10GbE ports. Also, make sure a gigabit ethernet port is connected to the independent server so we can download the software and a test suite. 
 
If you’ve not already downloaded Sniffer10G, then contact help@myricom.com and request a userid and password to access this product. Once downloaded issue the following command to install the product:
# rpm -i myri_snf-2.0.6.50271-2831.x86_64.rpm
 
This should put all the interesting stuff in /opt/snf. At this point, you should validate that you have purchased a card with the Sniffer10G license and that it’s ready to go. To do this issue:
 
# /opt/snf/sbin/myri_license
 
and it should return something similar to this: 
 
2 boards found
NIC 0 serial = 423691
License keys:
395e-e403-6a05-1a8f:2:423691:SNF:V2: # SNF, V2
NIC 1 serial = 423691
License keys:
395e-e403-6a05-1a8f:2:423691:SNF:V2: # SNF, V2
 
The reason it says two boards found is that each port is treated as a separate adapter. Next, you should confirm that the Sniffer10G code is running. The command to do this is myri_start_stop, you can use start, stop or restart. 
 
# myri_start_stop restart
Restarting Sniffer10G
Removing myri_snf
Loading myri_snf
 
Next, confirm that the system thinks things are ok by checking dmesg:
 
# dmesg | grep myri_snf
…
myri_snf INFO: 2 boards found and initialized
myri_snf INFO: eth4: Link0 is UP
myri_snf INFO: eth5: Link0 is UP
 
Now let’s confirm that traffic can pass easily between the ports. Here you’ll need two console windows open to your server, let’s call them Window-A and Window-B. On Window-A run the following command, it will setup your first port to receive traffic forever from the second port:
 
# /opt/snf/bin/tests/snf_simple_recv -p0 -t 1
snf_recv ready to receive
 
On Window-B run this command to generate one billion 60 byte packets on port 1:
 
# /opt/snf/bin/tests/snf_pktgen -p1 -s 60 -n 1000000000
 
Now in Window-A, you should see traffic along the following lines:
 
8650979 pkts (519058740B) in 1.001 secs (8641948 pps), Avg Pkt: 60, BW (Gbps): 4.148
14892539 pkts (893552340B) in 1.001 secs (14876784 pps), Avg Pkt: 60, BW (Gbps): 7.141
14894309 pkts (893658540B) in 1.001 secs (14878894 pps), Avg Pkt: 60, BW (Gbps): 7.142
…
 
Use Ctrl-C in both windows to stop things and in Window-B you should see:
 
Size Mbps Mpps Efficiency
60 7141.26 14.878 99.98%
 
This confirms that Sniffer10G is setup and working perfectly. Now on to Suricata.
 
Here are the steps to install Suricata, and build in the Sniffer10G libraries, note I’m not showing the output of the commands because it can be extensive:
 
 
# yum install file-devel
 
# tar -xvzf suricata-1.3.tar.gz
 
# mv suricata-1.3 suricata
 
# cd suricata
 
#./configure –with-libpcap-includes=/opt/snf/include/ –with-libpcap-libraries=/opt/snf/lib/ –prefix=/usr –sysconfdir=/etc –localstatedir=/var
 
# make
 
# make install-full
 
# cp classification.config /etc/suricata
 
# cp reference.config /etc/suricata
 
# cp suricata.yaml /etc/suricata
 
At this point we should confirm where Suricata is and if it was in-fact built with Sniffer10G. To do that use the following commands:
 
# which suricata
/usr/local/bin/suricata
# ldd /usr/local/bin/suricata | grep snf
libpcap.so.1 => /opt/snf/lib/libpcap.so.1 (0x00007f4359199000)
libsnf.so.0 => /opt/snf/lib/libsnf.so.0 (0x00007f4358b53000)
 
Now you need to edit your “suricata.yaml” file to make use of Sniffer10G. Suppose you have a dual socket system with eight cores per socket, you can then change your “suricata.yaml” file’s pcap section to look like this:
 
pcap:
– interface: eth4
threads: 16
buffer-size: 512kb
checksum-checks: no
 
We are assuming here that eth4 is your snf0 interface, you’ll need to confirm the actual interface names using “ifconfig”. Now to start Suricata and leverage all 16 cores, each with their own 512KB buffer use this command:
 
# SNF_NUM_RINGS=16 SNF_FLAGS=0x1suricata -c /etc/suricata/suricata.yaml -i eth4 –runmode=workers
 
For now, since the same system is both generating traffic and consuming it you should consider cutting this number back to 8 or 12 cores instead of all 16. During our testing, we typically use two systems connected back to back so we’ve not had to balance generation and consumption on the same system. 
 
This above command will start Suricata up, but the problem is where do you get traffic from. If you use Window-A for starting Suricata and Window-B for generating traffic then in Window-B we should first download some reasonable traffic. To do this use the following command from your home directory:
 
 
# mv 54 example.com-3.pcap
 
Then once suricata is running to generate test traffic you can use:
 
# /opt/snf/bin/tests/snf_replay -v -p0 -R 0.18 -i 2500 example.com-3.pcap
Thread 0> Packets: 5122500
Thread 0> Bytes: 1660497500
Thread 0> Rate: 0.27 Mpps
Thread 0> Throughput: 0.695 Gbps in 19.122 secs
 
Attached are three command output files:
 
 
and the two Suricata configuration files that were used for our testing:
 
 
These prove out the 2X performance improvement mentioned in the opening paragraph. It is possible to see gains well in excess of 2X, but those require additional tuning and tweaking of the suricata.yaml file beyond what was discussed above.

Leave a Reply