Many of us have been working in technology so long that we’ve become jaded, and readily dismiss what appears as new, yet minor advancements. Back in the 1990s, we were focused on the emergence of the web server, web site creation, and the DOTCOM boom. We barely took notice when Cisco released a router utilizing a new technique called Netflow. It quickly flopped as a method of routing, and gave way to Express Forwarding, but as is often the case it didn’t die. The new IPFIX standard is essentially Netflow v10 since it followed Netflow v9. Regardless, why has a technology that failed so early on, gone through nine subsequent major version releases? Primarily because network administrators wanted to know how their infrastructure was being utilized. Netflow morphed into a network accounting mechanism. The real benefit though has come in the past five or so years as it has risen in importance as solutions to mitigate advanced persistent threats (APT) have become critical. Packet capture is fine, but the volume of data retained can be crushing, often all you need is the intelligence captured in a Netflow record. So what are Netflows and why are they important?
In simple terms, a Netflow is a record of a connection between two systems during a period of time. Essentially a path of collected digital foot prints showing every detail of the connection between two systems. Netflow records, in fact, are so trusted that they are admissible in court as evidence. So what is a Netflow record? While Netflow v9 or IPFIX, the formalized standard derived from it, is the latest these are both pretty complex to digest this early on so we’ll look at the most commonly used version Netflow v5. There are two parts to a Netflow v5 event, the flow header, and the flow record. The flow header contains mainly time stamp information specific to the record when the record was generated down to the nanosecond, as well as some basic info about what generated the record. The record is where the real data lives. It contains the source and destination IP addresses, next hop router address, SNMP interface data, number of packets in the flow, the total number of Layer 3 bytes in the flow, TCP/UDP source and destination ports, protocol type, service type, and a series of flags. Records are generated when a flow is finished, or periodically for persistent flows. Whatever is exporting the flow can be configured to generate records periodically for open flows. A flow is considered finished when a TCP session termination is received or if the flow exceeds a predetermined age since the last packet in the flow was received. So what can these flow records be used for?
Have you ever watched a crime drama, and seen the police pull the suspect’s vehicle E-ZPass toll records or onboard GPS history. They are looking at real world flow data to determine if the suspect could have been on the scene of the crime at the time it occurred. In the cyber world Netflow, data would be the equivalent. It will tell you when an attacker connected with your system, how they connected, and how much data they stole. It will not tell you what specifically they stole, but knowing the time, how it was taken, and how much was stolen is often enough of a clue to enable you to explore other system specific records or logs to determine exactly what was taken. Looking at patterns of Net flows in near-real time (remember when net flow records are created) is a new technique for catching attackers before they can make off with your enterprise’s crown jewels.
Netflow data can we used for a great many things from balancing loads across a cloud application or infrastructure to tracking cyber crime or cyber warfare in near-real time. So how might you use Netflow to look for a cyber attack? One method is to employ the dark space within your companies Internet and internal corporate address space to setup decoy systems. The decoys could be simple stand-alone RaspberryPi systems configured to run Apache, Nginx, Memcached, MySQL, etc… and loaded with useless dummy data. These systems should typically see NO traffic because they serve no purpose to your employees or customers. Mark my words though, once they exist they will soon be discovered and interrogated. By monitoring the Netflows into these systems you can then use that source data as the key to search all the flows into all your production systems and desktops. Also, you could pro-actively use the data from these systems to strengthen both your perimeter and endpoint defenses.
Another method would be to look for Netflow patterns within your enterprise that are indicative of a cyber attack. All of the common penetration testing tools can be profiled in such a way as to yield a unique pattern of Netflows that they generate into a system they are preparing to attack. Often these tools work like a common home burglar doing reconnaissance before a job. If you can catch the burglar during his recon effort you can thwart his attack. Netflows provide you this data.
Currently, there is something of a gap in network speeds. More of a gulf, really. For larger businesses looking to invest in high-speed networking, the major choices at the moment are either 10Gbps… or 100Gbps. There is, obviously, a vast difference in price and implementation difficulty between these speeds, especially as 100Gbps is currently aimed only at the largest of enterprises and service-based providers.
A customer recently requested some 40GbE performance tuning advice, and this is what we shared with them.
This week 
What do you get when you put together a pair of dual socket servers running in hardware 
Proprietary technologies briefly lead the market because they introduce disruptive features not found in the available standard offerings. Soon after, those features are merged into the standard. We’ve seen this many times in the interconnects used in High-Performance Computing (HPC). From 2001 through 2004 
In two weeks I’ll be hosting two events: one near the NSA at the Courtyard Fort Meade on the 27th, and the second a few miles from the CIA at the Hilton Mclean Tyson Corner. Both will start at 4 PM, have a considerably strong Network Forensics agenda, and then wrap up by 7 PM. Both events are identical, they’re free to attend, and we’ll be providing beer, wine & soda along with appetizers.
Henry Ford is quoted as having once said “If I had asked people what they wanted, they would have said faster horses.” Not all innovations are as ground breaking as the automobile, but when one approaches an old problem with both a new strategy and improved technology great things can happen. In June Solarflare released an update to OpenOnload (OOL) that introduced TCP Delegated Send, and in late September they will begin shipping the latest version of their Application Onload Engine (AOE) with an advanced FPGA. The combination of these two will result in the capability of turning around a TCP Send in 300ns, compared to roughly 1,700ns today. In latency focused applications a savings of 1,400ns, an 82% improvement, is game changing. To understand how Solarflare pulls this off let’s look at a much simpler example.
Below is a document describing how to update/install Solarflare’s SolarCapture drivers on the SolarFlare 71xx NICs for a CentOS system. Note this document was user created and has not yet been reviewed by Solarflare’s Technical Support.
Today servers are typically installed with more than 16 logical CPUs, and yet many still leverage the same Gigabit Ethernet interface for production which was common when we had single core CPUs. There are three simple reasons you should consider 10GbE in your next server deployment: price, performance, and security.