Today cloud servers can host many Virtual Machines (VMs). When you add to that an advanced 10Gb Ethernet adapter, the adapter can become an in-server layer-2 network switch between all the hosted VMs on that server.  This enables two VMs on the same server to pass traffic between them using traditional network interface techniques without ever actually going out to a physical network. This technique is known as intra-VM switching.  A second interesting new performance feature is one called hypervisor bypass that enables the VMs to access the network via virtual functions (VFs) without involving the hypervisor after the connection is established. While both of these features can dramatically improve performance they also create new challenges. One of these is that by not having traffic going through a real network switch it short circuits some of the latest developments in Software Defined Networking (SDN). If you haven’t read much on SDN lately, it’s all the buzz in networking, and Bruce Tolley’s presentation “SDN: Science Project or Promised Land for the Networking Industry” frames the topic well. In these slides, he states that Dell’Oro forecasts the impact of SDN on the switch market going from $498M in 2013 to $2,200M by 2017.

Why might all this be important? Here is one example from a patent (#8,352,953) issued to IBM earlier this year for an application titled “Dynamically Provisioned Virtual Machines.” Four years ago IBM proposed a framework for software running on the network switch to keep track of how chatty all the VMs are on each node connected to a switch then to use this information to move the busiest VMs to least busy compute nodes. Pretty clever huh. With intra-VM switching through the network switch isn’t aware of any of the VM to VM traffic being passed within that server node so the switch never sees a full picture of what’s really going on.  Imagine if you will a classroom full of students. Suppose the students in this class were texting each other with their questions and answers, and that the teacher was only aware of questions asked of her.  If the traffic between students was high, and the teacher only had a few students raise their hand with questions then the teacher might come to some very wrong conclusions. So how can we fix this situation?

Turns out Solarflare’s adapters have the technology to address this. The first is Open Flow 1.0 control support. In a project with NEC, Solarflare provided an open API to create an Open Flow controller that enables an open flow mesh which includes the hypervisor resident vswitch. Furthermore, Open Flow 1.0 control support enables Solarflare to place a layer-2 overlay onto a layer-3 network to enable VM migration across the data center while preserving all the network links to the VM.

More on this topic soon in “SDN Within the Adapter Revisited”

At one point or another, we’ve all watched Trek. From Scotty yelling “I’m giving it all she’s got cap’n” to Picard’s “Make it so.” We love the mix of humanity & technology. Like Scotty, most techies get excited by the thought of pushing their own skills & systems to their very limits. Is there a place though in IT for pushing the tools we use to their limit, and can we overload our systems? I believe there is a place, and when it’s appropriate we should push our systems, and ourselves for that matter, to their limits so you can know for certain what we can truly expect. Computers these days have many safety systems to prevent us from frying them. Let’s face it, any techie worth his salt has the smell of burnt Silicon stored in the neurons of their olfactory nerve, so let’s try and avoid that. Yes, some of these systems can be bypassed, but today that’s not what I’m suggesting.

So how do we perform network analytics or problem determination on 10GbE links operating at near Warp speed? The answer is simple, get the right hardware, and software. You wouldn’t send a crew venturing off into deep space without at least a Warp 2 capable ship, so you shouldn’t expect to capture packets at high data rates on a 10GbE link with built in hardware and Libpcap. Heck, even my little $69 Raspberry Pi can be easily configured with Libpcap & Wireshark to analyze traffic.  No, to do real 10GbE performance monitoring and management you’ll need the right network interface card and software. Let’s assume you’re on a tight budget, and most folks are, my recommendation is an Emulex One Connect Network Xceleration (NX) adapter (P/N OCe12101DM-SNF2) which offers a single wire-rate 10GbE port WITH Myricom’s FastStack Sniffer10G included, all for under $690 list price. Now you might ask yourself, why not use something else? True there are other adapters and other solutions, although I think you’d be hard pressed to find another lossless wire-rate solution for 10GbE for one for under $700.

So what’s so special about this Emulex adapter and FastStack Sniffer10G? Several things: a transparent access model, lossless wire-rate 10GbE packet capture, multiple shared parallel memory buffers, two buffer creation strategies, adjustable hashing algorithm, user space or kernel mode, wire-rate injection, and a number of useful tools and sample code. Let’s take a moment and look into each of these.

By a transparent access model, we simply mean a libpcap replacement library on Linux, no coding required! This replacement library supports multiple in memory buffers & strategies that are totally transparent to your libpcap compliant application. Sniffer10G employs the use of environment variables for configuring in memory queues. This allows Sniffer10G to easily plug into programs like: Snort, Suricata, BRO-IDS, WireShark, TCPDump, etc…

Lossless 10GbE wire-rate packet capture requires three things: hardware designed for gargantuan packet rates, software architected to bypass the OS and quickly move packets into user space, and a tight marriage between the two. This Emulex adapter uses a processor made by Myricom capable of sustaining a packet handling rate of approximately 16 million packets per second (Mpps), combined send & receive.   A 10GbE link fully loaded with the smallest possible packets, 64 bytes, can sustain a wire-rate of 14.88Mpps. Often folks interested in packet capture typically don’t retransmit packets on the same link they’re capturing on so a ceiling of 16Mpps is a perfect match to catch 14.88Mpps of worst case inbound traffic.

Multiple shared memory buffers is a way to spread the work. FastStack Sniffer10G supports two buffer models: cloned & flow hashed. With cloning, Sniffer10G will create multiple in-memory copies of the same inbound traffic. This enables you to have multiple different applications running that each has their own independent in-memory copy of the packets being captured. At wire-rate, Sniffer10G should be able to sustain up to three clones for a total of four in-memory queues of the same identical captured traffic. Sniffer10G can provide more clones, but after three at wire-rate, we may begin to drop packets. Cloning brings up a significant problem though, each queue could potentially receive a new packet every 67 nanoseconds so your code that is working these queues has to be extremely tight.  Flow hashing solves this problem. With flow hashing, Sniffer10G spreads traffic between up to 32 different queues (each queue should be bound to its own core). This gives your code working the queue in a worst/best case up to two full microseconds (best meaning that the packet distribution is even across 32 buffers) to process inbound packets.

So what about flow hashing. Well, flow hashing is non-deterministic, meaning that it’s not a simple round robin method that yields easily predictable results because frankly, most folks interested in packet capture really don’t want this type of packet distribution between queues. Flow hashing allows you to maintain network flow affinity meaning that the same queue will always receive packets from the same source going to the same destination/port etc… so that when you do deep packet inspection for things that span multiple packets you’ll have access to all those packets in the same queue that make up that traffic flow. Pretty cool huh? So flow hashing is handled by default by using the IP/TCP/UDP source and destination addresses. This can be altered by setting the Receive Side Scaling (RSS) flags prior to launching your code using an environment variable (SNF_RSS_FLAGS).

The remaining cool features, such as an optional kernel space Sniffer10G driver, wire-rate injection & samples will be addressed in part two of this series… Stay tuned for Part 2.

In the 1970’s Intel brought us the CPU. During the 1990s we saw the evolution of graphics processors with Nvidia popularizing the term GPU in 1999. Now we’re witnessing the dawning of the Network Processing Unit, NPU. Much like it’s multiple core graphics cousins, the NPU is a parallel processing architecture, but it has been tuned for manipulating network traffic.

This market is still rapidly evolving, and several approaches are progressing in parallel paths. On one end we have Tilera, founded in 2005 by engineers from MIT and Broadcom, their approach is a many-core one, where the cores are interconnected by an on chip network to each other & substantial system I/O. In the middle we have Myricom founded in 1994 from Caltech, their architecture is multi-core where two buses are used by an intimate collection of cores to share on chip memory, I/O & multiple network devices which are more tightly coupled to the wire. Finally, there is the FPGA approach advocated by Napatech, eNdace, and Solarflare, here the focus is on using the FPGAs to provide well-defined filtering and packet processing. It’s not exactly a processor, but an interesting transitional step. Each of these approaches has value, but over the coming decade the market will decide a winner, it always does. First, let’s take a deeper dive into the raw hardware.

Tilera attacks the NPU much more like a GPU than the other two strategies. They pack up to 72 cores onto a single chip. Then Tilera leverages a network mesh architecture to connect the cores to each other, and to multiple: DDR3 controllers, Ethernet SerDes (ports), PCIe busses, and a pair of MiCA encryption acceleration engines. On the high end these 64-bit cores are clocked at 1-1.2Ghz, and typically each has both an L1 (32KB) & L2 (256KB) cache along with three execution pipelines. The mesh fabric in aggregate has over 100Tbps of bandwidth and utilizes a non-blocking cut-through routing with 1 clock cycle per hop. They then utilize four independent 72-bit DDR3 controllers for a total addressable real memory capacity of 1TB, and an access speed of 1,866 MT/s. On the ethernet side of this high-end chip, Tilera has eight 10GbE XAUI interfaces and 32 SGMII ports for legacy 10/100/1000 Ethernet. On the PCI Express side, they utilize six integrated PCIe controllers each with four lanes, providing up to 96Gbps of throughput. Finally, there are two MiCA acceleration engines, these provide for encryption support for six popular protocols along with a public key accelerator supporting four additional protocols. This is an awesome amount of hardware packed into a single 45mm square chip. The one-page data sheet doesn’t cover the power it requires, but it’s likely in the 60-80W range. How might you ask could I come to this conclusion? Tilera produces a quad-port 10GbE PCIe card utilizing the 36 core version of this chip, and it consumes 50W of power, it requires a secondary ATX power connector along with an active fan/heat sink.

As mentioned we have Myricom in the middle. Since 1994 Myricom has been producing a family of single core network processors for their line of programmable network adapters. In late 2005 they introduced their first 10GbE processor. Since then they’ve turned the crank once in early 2008 to produce their current Lanai Z8E. As mentioned this is a single core RISC processor clocked at 333+ Mhz with 2MB of on chip SRAM, two 10GbE XAUI interfaces, and single 8-lane PCIe controller supporting 8x, 4x, 1x modes. This summer it is expected that Myricom will deliver their first multi-core architecture chip, but no details have yet to be officially released. We do know though that it will be more similar to Intel’s multicore architecture than Tilera’s many-core interconnected via a mesh network approach. We also know they’re going beyond XAUI to more tightly connect to the wire to further reduce network latency, as this has been a key focus of theirs the past few years.

Finally, on the hardware side, we have the FPGA crowd headed up by Napatech, eNdace & Solarflare. Solarflare is the newest entrant in this space, but they appear to have the clearest vision of where this technology can really go. While Napatech & eNdace, now an Emulex owned company, have been focused on packet capture & network analytics Solarflare is focused on the High-Frequency Trading (HFT) market. Solarflare has taken their low latency 10GbE ASIC and back ended it with a powerful FPGA. Their intent is to keep fundamental packet processing decisions on the network adapter. Concepts like intelligent filtering and payload normalization. The filtering is based on some pattern of bytes within not only the header but also potentially the payload. If a packet makes it past the filter code then they are working to normalize the payload contents so trade data coming from multiple sources will all be formatted identically when passed up to the user space application.

So what challenges lie ahead for those interested in squeezing the most out of these NPUs? Programmatic ones. Tilera’s many-core approach and the wealth of hardware devices on the die are compelling, but frankly programming it is going to be like herding cats. The programmer will have to attach a pair of cores to each 10GbE port, one to handle receive and the other transmit. If we take the HFT problem described above that Solarflare is working on and use this as an example then they should utilize three additional layers of cores behind this. The first layer to handle filtering, say four cores to spread the filtering workload out. Each core would then be inspecting key strings within payloads for applicable securities symbols. The second layer would be data normalization, here another four cores would line up directly with the first layer. These cores would then be altering packet payloads to conform to a predefined standard structure. The third layer would be a single core to collect all the packets from the second layer and steer them to the proper user space memory locations via a PCIe device. So in this simple feed handler example, we’ve used two cores for the 10GbE link, one to interface with the PCIe and eight to handle feed processing, for a total of 11. For a single 10GbE link that’s a lot of cats to herd. On the opposite side, we have the FPGA guys, and we’ve all known for years that programming FPGAs is non-trivial. It’s gotten better, but it’s still more magic than engineering. Myricom being in the middle is still the wild card, as they’ve not yet said publicly how open their processing architecture will be.

So the NPUs are here, how will you leverage them to yield a competitive advantage for your enterprise?

In the 1990s the federal government pulled a page from da Vinci’s book and got behind a software acquisition strategy called COTS (Commercial Off The Shelf) expecting that it would dramatically reduce software development & deployment costs while improving overall quality. It didn’t take long before this approach began to be applied to application specific custom computing solutions used throughout key government agencies. Today many of these customers rely on systems built using a Lego like approach to building systems from COTS parts readily available to the general public.