Author: scottcschweitzer

I Can See You Now, in 10GbE – Machine Vision

scottcschweitzer networking
Who would think that something as fundamental as a video camera would ever need a 10GbE connection? Last week the tech who ran a well-known wind tunnel in the US contacted me to buy some 10GbE cards. I asked why, and he said because that’s the connector on the new cameras he was having installed, cool. At that point the whole concept of Machine Vision Acceleration (MVA), and the importance it might play as an emerging 10GbE market became interesting. During my research for this piece, I just learned that the Machine Vision (MV) market was $2B in 2011 and that the dominant standard is GigE Vision. GigE Vision was developed by a consortium of 12 companies and published in 2006, today over 50 companies produce GigE Vision products.

Automated manufacturing lines rely on cameras to capture images for analysis, and keep the lines moving. These digital images are then run through a number of recognition algorithms to determine if the products pass or fail based on certain quality standards: color, shape, orientation, etc… They can also be used to guide industrial robots. Technological advances since the development of GigE Vision have further accelerated adoption within this new market. Today we have 2-megapixel cameras that are capturing 340+ images/second, and actually doing some image preprocessing within the camera. These cameras have a built in FPGA (Field Programmable Gate Array logic chip) and significant memory. This approach enables one to write algorithms to preprocess the images, then load these algorithms into the camera to execute. This advanced processing trims down the data actually transmitted off the camera, while also distributing the processing load across all the cameras. It is not uncommon for a single server to process images from multiple cameras so if each camera can handle some basic takes this can dramatically ease the load on the server. Even with this assistance these cameras still require a built in 10Gb Ethernet interface for connection back to the servers that are processing the images in real time in order to properly direct the assembly line, robots, or security screeners.

Recently we covered HD video streaming. These machine vision cameras capture 2MP at 352 fps (Frames per Second). This is equivalent to between 6 and 12 uncompressed HD 1080p video streams (HD video can be between 24 and 60fps).One example I found of H.264 (the most widely used format) compression of a 1080p stream (1920×1080) requires 3Mbps at 0.4fps. So if we scaled this up to 352 fps this would work out to 2.6Gbps. Now you can see why these cameras have a built in 10GbE interface. The latest cameras are labeled GigE-Vision enabled devices, the new emerging standard. 
What networking magic could Myricom bring to MVA to improve performance? This is another market where bypassing the OS on the server side can make a dramatic improvement in overall solution performance. The processor on Myricom’s network adapter that resides in the server can detect a GigE Vision packet as it enters the server and directly places that image into the frame buffer of the GigE Vision enabled the application running on the server. Normally without MVA, it would take the OS on the server two copies, and all the associated CPU cycles, in order to pass the data to the application.  MVA reduces this to zero copies and no host CPU involvement. The second bit of magic that MVA brings to the table is intelligent interrupt moderation. With 10GbE running at line rate interrupt processing can seriously impact server performance. MVA utilizes adaptive interrupt moderation, known as coalescing, to reduce the impact on the server. With this technique, MVA only notifies the host CPU when the image is complete and waiting in the user space frame buffer.

The above two reasons, zero memory copies & intelligent interrupt moderation are MVA’s opening and closing illusions, but every good magician has several other tricks to fill out their act. Let’s face it flowers from a wand still get’s a laugh if properly timed. Another bit of slight of hand is how MVA manages load balancing and traffic steering. Normally the standard Ethernet or OS driver handles steering ethernet traffic steering to the application, MVA handles this now. MVA also supports multiple threads and devices and the steering between them. 

So all this magic, what’s the real payback, that Ta Da statement? How about a 25% CPU reduction in host CPU! That’s what all the above magic produces on an Intel 8-core 2.93Ghz system. Thanks to our Hawaiian surfing architect responsible for MVA for many of the details above.  
 
Today Myricom’s FastStack MVA is available as a technology preview. We’re still learning about this market, and tightening up what we think will eventually make for an awesome product in this space. So the next time you’re launching a Machine Vision project please consider taking a look at the technology preview release of Myricom’s FastStack MVA. 

Hey HFT, 4.9 Microsecond TCP Latency on Windows!

scottcschweitzer HFT

To all those High-Frequency Trading (HFT) shops out there that believe you need Linux to secure the lowest latency, well that’s no longer the case. Myricom & Emulex now bring you FastStack DBL for Windows Version 2.2 which boasts an impressive 4.9 microseconds for a half round trip. That’s a round trip divided by two which represents a send plus a receive. Oh, and that’s out of the box using a simple transparent mode wrapper (dblrun). If you want you can also code to the DBL API which now works for both UDP and TCP channels.

In April of 2011 at the HPC Linux for Wall Street show Myricom announced DBL 2.0 which included TCP, and Windows support. Since then we’ve learned a few tricks and have further reduced the latency in transparent mode to 4.9us. Initially, to achieve out of the box transparent mode we leveraged a technique called Layered Service Provider (LSP). We realized though that this trick cost us 500ns, and after considerably more research and coding have developed a non-LSP solution that saves the 500ns and has some additional performance improvements.

If you need additional flexibility, or require that DBL be in your code path along with other 3rd party modules, then DBL also offers a rich Application Programming Interface (API).

Furthermore, FastStack DBL now supports both UDP and TCP in transparent mode on both 32 bit and 64 bit run time environments in both transparent & API mode. So if you’re an HFT shop that uses Windows in production, or has been considering using it, now you’ve no longer got an excuse not to give it a try. All you’ve got to loose is latency.

"Simplicity is the Ultimate Sophistication" — Leonardo da Vinci (1452 – 1519)

scottcschweitzer networking

In the 1990s the federal government pulled a page from da Vinci’s book and got behind a software acquisition strategy called COTS (Commercial Off The Shelf) expecting that it would dramatically reduce software development & deployment costs while improving overall quality. It didn’t take long before this approach began to be applied to application specific custom computing solutions used throughout key government agencies. Today many of these customers rely on systems built using a Lego like approach to building systems from COTS parts readily available to the general public.

Many of these systems used by the military and intelligence communities require unique form-factors and may be mounted in hostile computing environments like the bed of a Humvee, on the flight deck of a C130 transport or the Command Control section of a nuclear submarine. There are a number of military specifications designed to address each of these harsh environments. Tracewell Systems produces a variety of systems specifically to meet these needs while housing COTS components to maintain flexibility. One platform utilizes IBM’s Bladecenter H blades unmodified in a three or five blade configurations that conform to several Milspecs. Since these utilize off-the-shelf IBM blades they can also have Myricom’s IBM Bladecenter H dual port processor based 10GbE mezzanine cards. These blades can then run Myricom’sFastStack Sniffer10G software to produce a very dense ruggedized enclosure supporting 60-100 Gbps of packet capture or injection capacity for network security or cyber warfare.
So what type of extreme performance ruggedized COTS solutions could one build utilizing a Tracewell Systems enclosure, with several IBM Blades each with a dual port Myricom 10GbE adapter, and FastStack Sniffer10? One example is a Highly mobile security appliance for Intrusion Prevention or Intrusion Detection handling up to 50Gbps of inbound Internet traffic. Several open source software packages like Snort, Suricata or Bro-IDS can be used on this platform to filter the incoming Internet traffic before it is passed on to the firewalls. Other tools are available for creating a defense against a DDoS attack. 
On the offensive side of cyber warfare, one could leverage the same hardware platform and the sample tools included with FastStack Sniffer10G to inject up to 100Gbps of traffic directly into the Internet. This can be done using previously recorded traffic or synthetically generated network packets. One or more of these systems could be racked into a single Humvee, driven to an Internet Exchange Point (IXP) anywhere in the world, and rapidly connected directly to the Internet via multiple fiber optic cables.
So the next time you explore military aircraft on display at a local air show, tour an army base or a naval shipyard it’s very likely that behind those custom RF shielded covers are COTS systems whose internal hardware is exactly the same as what you’re using today to run your business.

Arm photon Torpedoes. Prepare to fire on the Klingon Warbirds.

scottcschweitzer networking

Is an unforgettable quote from Captain James T. Kirk of the starship Enterprise that still sends chills down my spine. Unfortunately, we need to take a break this week from Machine Vision Acceleration (MVA), to let the ink dry on a few things. So instead we’re going to explore tools by answering one simple question, how strong is your optical signal? From mythical Photon Torpedoes to real life photonic communication mankind defines itself by its tools. How good are the 10GbE tools your NIC vendor provides to configure & diagnose the status of your adapter and optics, and are they free? 

Yesterday I had the pleasure of visiting a DoD customer of mine in DC whom I’ve been selling 10GbE NICs to since early 2006. They asked what appeared to be a very simple question: “How can we tell the optical signal strength coming into your NIC?” Their servers are installed in extremely remote collocation facilities. Clearly, this is a reasonable question, but in my nearly eight years of selling 10GbE, it’s one I’ve never been asked. I said I’m sure we have something that provides that info, perhaps “ethtool -S” might do it, I’ll look into it. It turns out we have a tool called “mdio_rw” that we provide with our free Myri-10G Adapter Tool Kit. Today I had a chance to install the toolkit and test it out. There are over 20 tools in this digital bucket, most are diagnostic or for extreme setup, meaning they aren’t for the faint of heart. Here is how you install the toolkit on Linux if you’re a windows user visit the above link for those details.

# gunzip -c myri-tools-linux.tar.gz | tar xvf – 
# cd myri-tools-1.26-linux/bin
Then to see the receive power, the amount of light arriving into the adapter on the first 10GbE port, you can issue the command:

# mdio_rw -b 0 | grep RX 

RX power = 0.6199 mW

This is the actual output from one of my test systems. The command returns another dozen parameters, some pretty interesting, but I’ll leave that to you to explore. 

Many of the 20+ tools in this package can be used to change how the adapter works by altering settings, upgrading the firmware, or performing deep diagnostics, all of these should be used with great caution. Here are several other interesting ones that rookies like I can play around with to gather useful information:


myri_info – Returns a number of hardware statistics on the adapter & PCIe bus. 
myri_pcie_conf – This confirms the PCIe payload sizes of the adapter and the system chipsets.
myrige-test.sh – Performs an adapter loopback self-test.

There is more to the loopback self-test, which you can read about in this link. 
So the next time you buy a 10GbE adapter check first to see what tools your vendor provides to help answer any questions you might have, diagnose any issues that may arise or allow you to further tune or configure your system’s performance. Also are the tools free, and easy to install and use? You may be surprised at what you might learn. 

Video Streaming to Jumbo Boob Tubes (Part 1 of 3)

scottcschweitzer networking

This series of articles was also posted on 10GbE.net. 

At CES this week Netflix, announced streaming support for SuperHD and 3D.  For a select few in the back of their booth, they also showed off UltraHD (4K) streaming. Some of you may know that Hulu Plus already supports HD streaming, but NetFlix is the 800Lbs gorilla, and when he moves people pay attention. Also this month we learned that Redbox has partnered up with Verizon to deliver their new Redbox Instant streaming service for the same $8/month that NetFlix charges.  Let us not forget that Amazon Prime also offers HD streaming, so there are plenty of choices now for streaming quality commercial content. 
Now, to be honest, most of my NetFlix streaming is still on my iPad, and the content is older TV shows that are likely only stored in Standard Definition (SD) format in the “cloud”.  Also like most folks these days my primary TV is one of those jumbo sets, but unlike my college sophomore daughter who exclusively uses a Roku box, ours is still connected to a digital Time Warner set top box.  So what does all this have to do with 10Gb Ethernet?  
Streaming content comes from service providers and content delivery networks that make up the “cloud”.  As we move from SD to HD formats like SuperHD, 3D, and soon UltraHD their servers and networks will be bursting with the new demand for bandwidth.  What is the actual impact of moving to these high-resolution options?
Here are some rough numbers that I’ve pulled from Wikipedia’s page on H264 stream encoding:

Format                Bandwidth   SD Equivalent  H264 Level

SD (480i)                  5Mbps              1X                 2.1
SuperHD (1080p)   25Mbps              5X                  4
UltraHD (3840p)   300Mbps             60X                5.1

How does this impact the video servers on the other side of the network providing your content? Well if they’re still using Gigabit (GbE) links, and only serving up SD streams they can support about 150 customers per network port.  When they move to 10GbE that jumps up to 1960 streams (10GbE links are a bit more efficient) per port.  Most of the Content Delivery Networks (CDNs) have already moved to 10GbE for this initial boost.  It becomes even more affordable when they utilize dual port cards so they can provide nearly 4,000 customers service from a single server.

Now along comes HD, and these servers with dual 10GbE ports can only handle about 800 customers.  Once those customers upgrade to UltraHD that number drops to 65 customers/server.  Wow from 4,000 to 65, and those 65 are consuming the same number of bits/sec so the backend storage demand remains constant. Server managers have to balance demand, people wanting to watch videos, with the hardware capability of both their storage & network infrastructure. As we saw on Christmas morning when NetFlix’s streaming service crashed for 24 hours, it’s not hard to miscalculate maintaining this balance.
Stay tuned, and next week I’ll share what technology is available to ease the burden on these video server vendors and server managers.

Video Streaming to Jumbo Boob Tubes (Part 2 of 3)

scottcschweitzer networking

The cult classic Monty Python and the Holy Grail in standard definition (SD) from Apple’s iTunes service is 1.00GB of data, the HD (720p) download file is 2.91GB, and the SuperHD (Apple just calls it 1080p) file is 4.84GB. This is a perfect real world example of the problem because the base file is something most of us geeks are familiar with and it’s exactly 1.00GB. As such it clearly shows the expansion as we move from format to format. Also, it should be noted that this doesn’t address UltraHD which will grow files another 12X in size over SuperHD.

So with the data demand for video streaming growing exponentially as a result of consumers moving from SD to HD & SuperHD, and soon UltraHD what can be done to ease the burden on these servers? First, we need to take a quick low-level look at the problem. A movie in the “cloud” is nothing more than a file on storage somewhere that needs to be transferred through a network to a consumer. Focusing on the server side we have several areas where one can further optimize performance:

  • Tracking all the clients & their associated streams.
  • Moving the bits that make up the movie from storage to memory.
  • Then moving those bits from memory to the network adapter.
  • Finally transmitting these bits from the network adapter onto the wire so they can reach the consumer.

Tracking network connections and mapping them to streams is a collaboration between both the application, and the OS. File transfers from storage to memory is an OS function requested by the application, these have been well optimized over the years so there can only be marginal improvements here. Moving bits from memory to a network adapter is also considered an OS function, but it’s been one that the HPC (High-Performance Computing) crowd has been optimizing for well over a decade through a technique called OS Bypass. This is still more art than engineering so it’s not mainstream. The final step is often considered hardware, but it’s also the most intriguing. Normally packets arrive via the OS and have to be transmitted as is onto the Ethernet, boring. If on the other hand, they arrive via OS Bypass then some real magic can happen.

It appears that two of the highest profile web video streaming entities, NetFlix & Hulu, along with a collection of others have moved to a new web server platform called Nginx (pronounced engine-x) to address the C10K problem. C10K is the name given to a class of problems associated with keeping track of 10’s of thousands of web server clients, and the data they are connected to. Nginx addresses this through a number of sophisticated techniques, that are better explained on their website.

The next issue is that of transporting the movie from storage, as a file, to memory. Again, this falls into the realm of Nginx as it’s both the application and the OS that are responsible for all the latency around file I/O. Nginx has had extensive tuning over the years to minimize this sort of latency.

So now we’re left with what is considered both an OS and a hardware issue. This is moving the bits to the network adapter and putting them on the wire to the client. Nginx with any generic 10GbE adapter works fine, but the real magic happens when it’s coupled with an intelligent 10GbE adapter and FastStack™ VideoPump™ which utilizes HPC OS Bypass techniques to dramatically improve system performance. Next week in part 3 of this series we’ll show you what those improvements really are and give you a peek into the magic behind the curtain that makes this all possible. Stay tuned, same bat time, same bat channel…

Video Streaming to Jumbo Boob Tubes (Part 3 of 3)

scottcschweitzer networking

So how do web content delivery networks (CDN) or a video streaming service providers like NetFlix or Hulu handle such explosive growth? Well, the first thing most of them did was move to a more robust web server platform designed to handle many thousands of users at a time, in most cases as mentioned in “Part 2” that has been Nginx. As content, Video places a far greater demand on the server than traditional web pages. A normal web user requests a page, the components arrive, they are then assembled by the user’s browser, and typically read for about one minute. The user then requests another page. A video user, on the other hand, is far more demanding. For their experience, they demand a constant stream of packets is sent to them continuously for very long periods of time. These packets need to be transmitted from the content server at very regular intervals, much like the lights on California highway on ramps that meter traffic entering the highway during rush hour. Maintaining continuous streams of video packets to thousands of web connected users is something Nginx can do very well.

So we mentioned in “Part 2” of this series that magic can occur when Nginx is coupled with an “Intelligent NIC”. This “magic” is called FastStack VidePump™, and here are three main benefits:

Congestion mitigation, having a well-tuned user space communications stack and a processor based NIC enables us to be considerably smarter with regard to sending many streams through many small pipes.

Asynchronous sendfile() support, this dramatically improves overall I/O performance.
This “magic” requires a processor based network adapter that can communicate directly with user space memory. This alone dramatically offloads the host CPU from having to provide file based content delivery to web users.

FastStack VideoPump™ offered with both Emulex OneConnect and Myricom network adapters provides all of the above features. Today Nginx supports many users by typically having 1,024 worker threads all doing synchronous sendfile(). With VideoPump™ and a single worker thread we see dramatic performance improvements because Nginx isn’t flopping around between many threads trying keep the I/O pipeline full. So if all of the Nginx video streaming demand is coming from TCP sources (intelligent devices like browsers or applications running on laptops, tablets or smartphones) and the video content is being served up from files, the performance gains afforded by VideoPump™ in transparent mode (which requires no code changes to Nginx or applications) delivers a 25% reduction in host CPU utilization.

VideoPump™ can also be accessed through an Application Programming Interface. For applications designed to serve video content we’ve measured host CPU gains of up to 50%. To learn more about FastStack VideoPump™ please consider contacting your Emulex or Myricom sales representative.

A Fire Sale, Napolitano, and Cyber 9/11 is it Probable or Just Possible?

scottcschweitzer Security

This article was originally published in January of 2013 on 10GbE.net. 

We’re going to zero in on a low profile Cyber Warfare story that I’m surprised saw such little media coverage. Thursday U.S. Secretary of Homeland SecurityJanet Napolitano talked publicly at the Wilson Center (a DC Think Tank) on the looming threat of a “Cyber 9/11” with the potential for a hurricane Sandy like outcome. Wow, now that’s a current image that evokes fear. When I first learned of the story Friday morning my initial impression was that perhaps earlier in the week Secretary Napolitano had picked up Bruce Willis’s 2007 movie “Live Free or Die Hard” in the discount bin at the Walmart and watched it a bit too late one night. The skeptic who brought this story to my attention pointed out that it was perhaps a veiled attempt to grab more budget and create even more regulation & government oversight. 

Now I’m not just some random blogger with an opinion, and no street cred to back it up. I started hacking back in 1983 as a hobby with my TRS-80 Model III when it was still cool, and legal. “War Games” had just been released, and ethics were the only rules. Later that year I was hired by IBM Research, a company I remained with for 16 more years. During that time I’d tied many different systems together. At one point I worked side by side writing code for Davis Foulger to link Prodigy, Compuserve, AOL, and IBMPC (the IBM internal social network) together so IBMers could answer customer questions from anywhere using a single internal system. In the late ’90s I designed and created a solution for the company that linked many different systems together and in 1999 it collected $2B in revenue. This effort earned me an IBM Outstanding Technical Achievement Award, but to pull this off we also created a state of the art hacking tool for Lotus Domino to certify that our system was secure, which almost landed us in jail. So I know a bit about cyber security, and linking together different systems.
 
So is a “Fire Sale“, a “Cyber 9/11”, probable or Just possible? Possible, definitely. Probably, within a reasonably defined scope, say a single city, and it would require substantial capital to motivate and assemble a team with the proper skills. The team would need to touch every power grid, water, sewage treatment, mass transit, oil refinery, financial trading, etc. system within the scope. These systems don’t need to be Internet connected, people are always required to service, and inspect them, and internally they are often networked together. One would simply have to crack the case off a $100 Verizon 4G USB Cellular modem, then 3D print a new case branded to match the software product that industry runs so it would be camouflaged as a USB software license key. It could go unnoticed forever unless they did an active RF sweep. Prior to inserting this USB stick another containing a modified version of Operation Olympic Games (Stuxnet) could be used to open the security hole and inject the software, then the 4G connectivity would be used to continuously report back, and eventually carry the trigger message to bring the system down. The right mix of social hacking, posing as contractors doing repairs, or government regulators conducting inspections, even OEM system engineers doing required service could easily facilitate this objective.
 
George W. Bush authorized Operation Olympic Games in 2006, during Homeland Security Secretary Michael Chertoff’s time in office, why wasn’t this considered and addressed seven years ago? Secretary Napolitano has been in office three years now, and all of the sudden it’s a priority. Our government could form its own cyber security Tiger Team. This team could then covertly attack key industries, and report back to these firms their findings, and the appropriate counter measures. These attacks would simply demonstrate vulnerabilities, not bring down systems. Since the early ’90s, IBM has had it’s own Security Tiger team, “Global Security Analysis Lab“, that regularly trained key three letter agencies. I met with their leader in the late ’90s to provide him with a copy of our IBM Internal use only Lotus Domino ethical hacking tool.
 
A “Cyber 9/11” can be prevented, but regulations aren’t the answer, that process is far too slow and rigid. An elite group of patriotic, well compensated, cyber professions could do in two years what the administration has ignored in the past seven years. Only time will tell.

How Ethernet Won the West (Part 1 of 2)

scottcschweitzer networking

This article was originally posted in December 2012 at 10GbE.net. 

Back in the early 90s an internal IBM cartoon called “How OS/2 Won the West” was being passed around on VHS it was hysterical but ultimately wrong. Superior technologies and products like Betamax, Token Ring & the Apple Newton litter the battlefield of technology. In the end, all that matters is market dominance.  It’s our culture, look at the Olympics, we all know that Gabby Douglas won gold for the Women’s Gymnastics Individual All-around, so who won the silver?  Someone from China? No, Viktoria Komova from Russia.  Our society pays very little attention to second place.  

Over its three decade rule of the networking market Ethernet has crushed: PC Network, Token Ring, AppleTalk, DECnet, Myrinet, Quadrics, and dozens of other interesting approaches.  Today only two competing technologies remain and each has tried to carve out its own niche.  Fiber Channel defined and ruled the storage market, while Infiniband has become the network of choice for the scientific community.
 
Fiber Channel is now seriously threatened by a protocol called FCoE, Fibre Channel over Ethernet. Essentially all that is Fiber Channel is wrapped in an Ethernet package and shipped over an Ethernet network.   Fiber Channel has moved through  2, 4, 8 and now 16Gbps.  During roughly that same time Ethernet has migrated from 1 to 10 and very soon 40Gbps (note 100Gbps exists for the switch to switch connections, but it is impractical today for the server to server communication).  Much of the Fiber Channel world is running with 4 & 8Gbps networks while the server market has been moving rather quickly from 1 to 10Gbps. Furthermore, Ethernet controllers, the chips in the server that connect to the actual network, have gotten far more sophisticated over the last two generations so they can now off-load the host CPU from doing many of the mundane networking chores while delivering amazing performance.  Alan Weckel, senior director at Dell’Oro, said “We believe that in 2013, most large enterprises will upgrade to 10Gb Ethernet for server access through a mix of connectivity options, ranging from blade servers, SFP+ direct attach and 10G Base-T,”  Alan then went on to say “We further anticipate that in 2014, small and medium businesses will upgrade to 10Gb Ethernet.” The bleeding edge customers will begin testing 40Gb Ethernet in 2013, with adoption really starting to ramp in 2014.  At this point several things converge:  Ethernet network performance will far exceed Fiber Channel network performance, this generation of ethernet controller silicon coupled with enhanced value added software should be mature enough to deliver all the features of Fiber Channel, and all of this will be done at a commodity price point.
 
Infiniband (IB) is a network of a whole different color.  Here the value proposition is two fold, low latency at a low price point.  One of the issues plaguing IB is the decreasing number of competitors in this market that sells switches and adapters.  Several years ago there were well over a half a dozen, today there is just Mellanox and Intel (they bought QLogic’s switch & adapter business in 2012 for $125M).  Intel’s roots in this space run deep, all the way back to 1999.  At that time Intel had co-developed something called Next Generation I/O (NGIO) with SUN and Dell.  In 1999 a consortium was formed that merged NGIO with Future I/O (a competing technology developed by IBM, HP & Compaq) to become Infiniband.  The division of Intel that acquired IB from QLogic is the same division that drives Intel’s very robust 10Gb Ethernet business.  In 2011 they acquired Fulcrum to fill out their networking line in order to compete head to head with Broadcom.  This division of Intel did just over $10B in 2011 with expectations of ramping it to $20B over the next five years. Intel, like Broadcom, is very good at selling chips, and pretty good at selling adapters.  I doubt we’ll see any new Intel developed IB switches anytime soon so Mellanox will likely be walking this path alone. It will be interesting over the next year to see if Intel fuels IB or pours water on it.
 
Next week – Part II, “Where is Ethernet Headed & How Network Companies, Other Than Intel & Broadcom, Will Survive & Prosper”

How Ethernet Won the West (Part 2 of 2)

scottcschweitzer networking

This article was originally published in January 2013 at 10GbE.net. 

This could also be titled where is Ethernet headed & how network companies, other than Intel & Broadcom, can survive & prosper.

Last month we saw two interesting announcements that may have foretold the future of Ethernet.  The first was by Emulex, and it was that they were going to be acquiring Endace, an Ethernet analytics technology company and that it would close this acquisition in the first quarter of 2013.  The second came on“Cramer’s Mad Money” when Cramer said sell Mellanox, don’t touch the stock.  It has dropped from a high of $120 in September to $50 this week, a nearly 60% drop in three months.
 
In May of 2012, Emulex partnered up with my employer, Myricom, to license both our hardware and software technology for adding value to Ethernet.  Two years ago Emulex acquired ServerEngines, a company that had commodity grade 10Gb Ethernet adapter silicon.  So why did a leader in Fiber Channel storage networking buying a 10Gb Ethernet commodity silicon company, partner with a value added Ethernet company and announce an intent to acquire a third Ethernet analytics company?  Simple Emulex understands that Fiber Channel has the same future that Token Ring, AppleTalk, and  Myrinet have all shared, and that’s Ethernet.  Ethernet isn’t the best architecture, but it is by far the most pervasive.  The IEEE, the standards body behind Ethernet has remained active so the standards continue to evolve and keep pace with processor technology. If you recall from your days back in Computer Architecture, there are four performance curves: CPU, memory, disk & network.  When we moved from Gigabit Ethernet to 10Gb Ethernet the network curve jumped in front of the disk curve.  Now even disks spinning at 15K RPMs can only provide data at about 1.2Gbps, and the newest FLASH drives are typically only 4X better, still well under 10Gbps. Furthermore, 40Gb Ethernet is just around the corner for server to server communications.  Add to that enhancements like: Data Center Bridging (DCB) which includes key concepts from Converged Enhanced Ethernet (CEE) and Data Center Ethernet (DCE),  RDMA over Converged Ethernet (RoCE, pronounced Rocky), and the Common Communication Interface (CCI) and you’ll see that Ethernet is continuing to expand to meet the demands of the bleeding edge. Companies like Emulex understand that the future of networking is Ethernet, and that profit will only come by adding value on top of this industry standard.
 
With only two companies selling Infiniband the future of this interconnect is in question.  On August 20th Cramer’s Mad Money was Obsessed with Mellanox at $113, it soon hit $120.  At the time the implications of Intel’s acquisition of QLogic’s Infiniband assets weren’t totally understood.  Intel is a whale in the silicon market, and it can swish its tail one way and buy Fulcrum (Ethernet switching silicon), then the other and buy up the second to last Infiniband provider.  The question is will they continue to support both?  Now four months later Cramer’s calling Mellanox “the disaster du jour” as it struggles in the $50s.  You can bet Cramer’s recent remarks aren’t going to help that stock move up anytime soon. Efforts like CCI, mentioned above, will remove most if not all of the technical advantages Infiniband might have, and this will force Mellanox into committing 100% to becoming an Ethernet technology provider. At that point on the server side, they’ll need to compete head to head with Emulex, and Qlogic who are both very well entrenched, and on the switch side with a host of deep pocketed competitors like Cisco, Brocade, Foundry, Force10, Extreme, etc… Competing on two fronts against well-established companies as one orphan their primary technology differentiation is a challenge very few companies ever have to face, let alone survive.
 
So what does all this mean?  If you are a networking company it’s rather simple, embrace Ethernet, and find a way to add value on top of it or die.  I’m sure other networking technologies will pop up in the future, like say quantum state photon based approaches, which will spawn new companies, and network products. Eventually, though I’m sure we’ll figure out how to map Ethernet into the quantum world…
 
Happy New Year.