World War III is Now in Cyber Space

This article was originally published in May of 2011 on

“We don’t know who struck first, us or them. But we do know it was us that scorched the sky.” Morpheus explaining to Neo how the final World War between the humans & the machines started.
Unlike the first two World Wars, or the final fictitious one is in the Matrix. WW III is being fought today exclusively in cyber space.  It is a silent war, rarely reported in mainstream media, primarily because there’s almost never a human casualty*, and most readers wouldn’t understand it.  Often the combatants have no idea who they’re fighting. If we look at one of the early major battles, Stuxnet, no one has officially claimed to have launched the attack, but the target was laser focused on Iran’s nuclear weapons program. The campaign was successful, and it delayed Iran’s weapons program by several years.
Now I’m sure you’re wondering, “Why is this being covered in a blog about 10GbE?”  Simple, all battles require supply lines. Today the supply lines for cyber warfare are moving from GbE to 10GbE.  Sure a lone hacker can use a Low Orbit Ion Cannon (LOIC) to take down an unsophisticated opponent using little more than a laptop, and a decent Internet connection. Frankly, this is synonymous in the real world to someone with a machine gun trying to shut down a store.
On the other hand governments, and corporations have substantially more resources.  As we saw with Stuxnet.  For those not familiar with Stuxnet, this was a very carefully engineered offensive cyber weapon delivered in virus form.  It had several different parallel attack vectors to ensure that it reached all the intended targets, which reports suggest that it did.  It contained a precise triggering system and a lethal warhead.   This battle would not have been possible with only a handful of intelligent people.  It took a collaborative effort by several governments, at least one corporation, and several unique pieces of equipment for testing to ensure the weapon’s success.  This went way beyond your garden variety LOIC class assault and defined a whole new category of offensive cyber weaponry.  While the LOIC is a machine gun obtainable by nearly anyone, Stuxnet, by comparison, is a state of the art cruise missile.
Now back to 10GbE.  Markets like HFT and HPC are breaking new ground in low latency, high packet rate & wire-rate bandwidth.  They are utilizing advanced 10GbE network adapters and switches, along with sophisticated algorithmic routines.  Today HFT’s are exploring lossless wire-rate 10GbE capture to do real time analysis and simulation.  They capture real trading data in buffers or spin up synthetic ones to emulate the market.  They then inject these buffers into an HFT trading engine to see how it responds.  Is this really any different than simulating 100s or 1,000’s of web surfers?  This technology is crossing over into cyber warfare.
All of the above was written over the holiday weekend.  This morning the Wall Street Journal reported that the US Government has announced that The Pentagon has adopted a new strategy that will classify major cyber attacks as acts of war, paving the way for possible real world military retaliation.  “If you shut down our power grid, maybe we will put a missile down one of your smokestacks,” said a military official as quoted in the WSJ.  So if you don’t believe the ongoing war in cyber space is real, then just consider it a prelude to it.
* John P. Wheeler III may very well be the first person slain in this cyber war.  He is a former Pentagon official and was a consultant for Mitre prior to his murder.  John was an outspoken proponent of Cyber defense.  In January 2011 he was found dead in a Maryland landfill, the investigation into his death is currently stalled.

Optical Lock Down

This article was originally published in May of 2011 at

Today for the umpteenth time I had to explain to someone that if you go optical to connect your server to your switch with 10GbE it could easily cost you twice as much.  There is a secret at the end of this entry that MIGHT allow you to save some big time cash if you have enough muscle, but you have to read to the end of this entry.

For cable runs of seven meters or less you should always use Direct Attach (DA  otherwise known as Twinax) cable if possible as it could easily save enough to basically connect the second server for free! Here are some actual numbers from earlier today.
First, some basic end user costs assuming a five-meter run, note these are rounded a little bit to keep the math simple:
10GbE Network adapters, roughly $400/port
10GbE Switches, roughly $500/port
10GbE SR SFP+ Optics from switch vendor $800/port
10GbE SR SFP+ Optics from NIC vendor $200/port
10GbE SR Optical 5M cable $80/ea
10GbE Direct Attach 5M cable roughly $140/ea
Now let’s build a solution between the server and the switch using optics:
10GbE Network adapter $400
10GbE SR Optic from NIC vendor $200
10GbE SR Optical 5M cable $80
10GbE SR Optic from  Switch vendor $800
10GbE Switch port $500
Total $1,980 to connect a single server
Direct Attach (Twinax) Option:
10GbE Network adapter $400
10GbE Direct Attach 5M cable roughly $140/ea
10GbE Switch port $500
Total $1,040 to connect a single server
Let’s look more closely at the market dynamics going on here.  First, only a handful of companies make 80% of the 10GbE Short Range (SR) optics that everyone uses today.  These companies are typical: JDSU, Finisar, Agilent, etc…  None of the switch companies or NIC companies make their own optics, we all source them from several of the above companies, and a few others, all of whom rebrand them for us and burn our company name and part number into what is essentially flash memory within the optic.
Here’s where it gets interesting.  Myricom, the company I work for, sells it’s SR SFP+ optics online via CDW’s website for $185.  Here are some of the more expensive SR SFP+ optics listed on CDW’s site:
HP Procurve: $1,498
Avaya: $1,350
Enterasys: $1,210
Cisco: $1,100
Juniper: $1,082
Brocade: $1,022
QLogic: $930
IBM: $920
Now remember under the covers we’re all sourcing these optics from the same competitive pool, so why the price spread?
First, remember that we each buy our optics with our manufacturer name and part numbers already burned into them by the optics makers mentioned above.  Now here’s where it gets interesting the switch makers during switch initialization query the optic and if it does not return a valid company name and part number then it locks the optic out and reports the port as offline.  
A Cisco switch requires a Cisco optic.  If you were to use a Myricom optic it would see that the optic was made by “Myricom” with a part number “10G-SFP-SR” and it would lock that port out because it has an incompatible optic.  Never mind that a valid Cisco optic and the “failed” Myricom optics may very well have been made by JDSU on the same assembly line, perhaps even on the same day. 
Network adapter vendors, like Myricom, are optic agnostic. You can shove in an Arista, Cisco, HP, or Gnodal, we won’t care.  We provide optics to offer a complete solution for our customers.  Finally, we are not “in the optic business” so we pick them up, mark them up fairly, then offer them for sale.  I can assure you we’re not buying them at the same discount that a Cisco or Juniper might be getting, yet our price is clearly so much more reasonable.  
Now here’s the secret I promised.  Most switch vendors have a patch for the switch operating system so that it will ignore the optic check and allow you to use anybody’s optics.  If you have the buying power and the cojones, then insist that they provide the patch as a condition of buying their switch.  It will save you big time.  You can then take those savings, and buy a few more Myricom 10GbE adapters.

Dualies Aren’t Just for Trucks

This article was originally published in April of 2009 at

One would think that after 30 years our industry would have developed a NIC naming convention for “dual-port.” Does a dual-port NIC mean your OS sees one or two interfaces? Do dual-port NICs mean that one port is active and the other is for fail-over? Can a dual-port run traffic through both port simultaneously? It all depends on who you talk to, and the product they’re selling.

With 10GbE we’ve seen three main approaches for building dual-port NICs:
Active/Active: this is what most people expect, a single OS interface with a driver that sprays traffic fairly evenly across both network ports and if one port fails the other picks up the slack until it can handle no more:
  • Chelsio’s N320E for $790 is an example of this type of card.
  • Intel’s AF DA card for $799 appears to be another example of this class of card.
Dual-NIC: two OS interfaces are presented to the OS and both interfaces run independently. This typically affords the best performance and the most flexibility:
  • Myricom’s 10G-PCIE2-8B2-2S+E for $995 appears to be the only example of this approach. Myricom utilizes two unique 10GbE controllers on the same PCI Express Gen2 NIC and a PCI Express bridge chip to break the slot into two unique NIC devices.
Active/Passive or Active/Fail-over: a single OS interface with a driver that monitors connectivity on the active port and if the connection fails the driver migrates traffic rapidly over to the second port:
  • Myricom’s 10G-PCIE-8B-2S+E for $795 is an example of this type of card. The fail over time is under 10 microseconds.
  • Chelsio’s B320E Bypass adapter for $3,483 is similar but it can detect an OS/BIOS/System failure and make a hard switch over to the second port.
Do the above categories cover it, or do we need more lingo? When looking for a dual-port NIC, what features do you require, and what do you expect? Please let us know.
P.S. As I brought this page back online I left off the links as most no longer apply, but from a historical perspective it is interesting to see how things have progressed.

Thinning the 10GbE Herd

This article was originally published in January of 2009 at

Thinning the herd.

In 2007 over one million 10GbE network ports were purchased. Many of those were for a switch to switch interconnects but some were to connect servers to networks via 10GbE. Natural selection is now taking effect in the 10GbE NIC market as the big dogs, Intel & Broadcom, start thrashing around in an effort to secure market share as 10GbE matures. Both want to dominate the 10GbE LAN on Motherboard (LoM) market. In the NIC market, four companies likely supply over 80% of the 10GbE NICs purchased and they are Chelsio, Intel, Myricom, and Neterion. The remaining 20% of NIC sales fall to companies like Broadcom, SMC, NetXen, ServerEngines, Tehuti, AdvancedIO, Endace, Napatech, etc… One should be wondering why Broadcom is in the second group, it’s because Broadcom’s focus is on selling 10GbE silicon to OEMs like IBM and HP for LoM projects positioning their silicon on high-end server mother boards and not retailing NIC cards. 

Officially the first documented victim is NetEffect, the leader in iWarp (Infiniband for 10GbE) NICs. NetEffect rose from the ashes of a failed Infiniband company, Banderacom, earlier this decade to apply their silicon development skills and Infiniband algorithms to the more stable Ethernet market as a new feature called iWarp. NetEffect in-fact led the iWarp charge, it was the self-proclaimed leader in low-latency iWarp 10GbE NICs. In August NetEffect filed for reorganization in US Bankruptcy Court. With the failure of NetEffect the market has cast its vote and drove a stake through the heart of iWarp, hopefully terminating this feature.
Rumors have been swirling around Teak Technologies, a maker of 10GbE NICs and a switch, for some time. It appears that Teak has not weathered the storm and has since faded away, their domain name is no longer resolving to an IP address. The domain was never transferred from the founder, and the founder announced this spring on Linkedin that he had moved on some time ago. Is it conclusive evidence, no, but would you buy technology from a tech company whose URL won’t resolve to a server?
It is a tough economic climate for start-up NIC companies, particularly those in the bottom 20% as they have likely never had a quarter in the black. Now is a challenging time to be out there seeking another round of capital from ones VCs. Several have been without an injection of new funding for over two years and lack the sales volume required to sustain their own existence much beyond year end. As such we’ve directly questioned one firm to see if they are alive, and another that is widely rumored in the industry to be in trouble, but their marketing departments are still bailing.