My first co-op job at IBM Research back in 1984 was to help roll out the IBM PC to the companies best and brightest. It wasn’t long into that position, perhaps a month, when we noticed a large number of monochrome monitors had a consistent burn-in pattern. A horizontal bar across the top, and a vertical bar on the left. Now I was not new to personal computers having purchased my own TRS-80 Model III a year earlier, but it was apparent that a vast number within Research were all being used to do the same thing. So I asked, VisiCalc (MS Excel’s great-grandfather).

At the time personal computers were still scarce, and seeing one outside a fortune 100 company was akin to a unicorn sighting, so the subtlety between tool and solution was lost on this 21-year-old brain. Shortly after the cause of the burn-in was understood I had the opportunity to have lunch with one of these researchers and discuss his use of VisiCalc. That single IBM PC in his lab existed to turn experimental raw data that it directly collected into comprehensible observations. He had a terminal in his office for email, and document creation, so this system existed entirely to run two programs. One that collected raw data from his lab equipment, and VisiCalc, which translated that data into understandable information which helped him make sense of our world.

Recently Solarflare began adding Analytics packages to their open network packet capture platform as they ready it for general availability later this month. The first of these packages is Trading Analytics, and it wasn’t until I recently saw an actual demo of this application that the real value of these packages kicked in. The demo clarified for me this value much like the VisiCalc ah-ha moment mentioned above, but perhaps we need some additional context.

Imagine a video surveillance system at a large national airport. Someone in security can easily track a single passenger from the curb to the gate, without ever losing sight of them, now extend that to a computer network where the network packets are the people. The key value proposition of a distributed open network packet capture platform is the capability to gather copies of what happened at various points in your network while preserving the exact moment in time that that data was at that location. Now imagine that data being how a company trades electronically on several different stock exchanges. The capability to visualize the exact moment when an exchange notified you that IBM stock was trading at $150/share, then it updated you every step of the way showing you the precise instant that each of the various bits in your trading infrastructure kicked in. The value of being able to see the entire transaction with nanosecond resolution can then be monetized, especially when you can see this performance over hundreds, thousands or even millions of trades. Given the proper visualization and reports, this data can then empower you to revisit the slow stages in your trading platform to further wring out any latency that might result in lost market opportunities.

So what is analytics? Well, first it’s the programming that is responsible for decoding the market data. That means translating all the binary data contained in a captured network packet into humanly meaningful data. While this may sound trivial, trust me it isn’t, there are dozens of different network protocol formats and hundreds of exchanges worldwide that leverage these formats so it can quickly become a can of worms. Solarflare’s package translates over two dozen different network protocols, for example, NYSE ARCA, CME FIX and FAST into a single humanly readable framework. Next, it understands how to read and use the highly accurate time stamps attached to each network packet representing when those packets arrived at a given collection point. Then it correlates all the network sequence ID numbers and internal message ID numbers along with those time stamps so it can align a trade across your entire environment, and show you where it was at each precise moment in time. Finally, and this is the most important part, it can report and display the results in many different ways so that it easily fits into a methodology familiar to its consumer.

So what differentiates a tool from a solution? Some will argue that VisiCalc and Solarflare’s Trading Analytics are nothing more than a sophisticated set of fancy hammers and chisels, but they are much more. A solution adds significant, and measurable, value to the process by removing the mundane and menial tasks from that process thereby allowing us to focus on the real tasks that require our intellect.