On June 8th, 2017 Ron Miller of Cloudwick joined us to go deep on Hadoop and Securing Hadoop Clusters. Ron is a member of the CTO staff at Cloudwick, and he’s been an industry leader in security for over two decades. I was fortunate enough to work with Ron for several years while he was with Solarflare, and we’ve remained friends since his departure. In 2015 we had the opportunity to work Black Hat and attend DEFCON together, and it was amazing to hear his perspective on the security products and events.
5 Petabytes of Public Hadoop Data
Back in 2015, I sat through a DOE presentation during a government cyber security conference on SCADA (Supervisory Control and Data Acquisition) systems accessible from the web. SCADA is used to allow computers to manage public utilities, water, gas, petroleum refineries, nuclear power plants, etc… The speaker did a live demo using Shodan where he was able to demonstrate something like over 65K open SCADA networks reachable from the Internet. This article backs up the above-mentioned presentation, though the author points out that the maps only show German made SCADA systems. To be more precise the maps show Seimens SCADA controllers, which dominate the market. Most of these systems were for industrial control, and they should have been air-gapped, physically not connected, to ANY external network, let alone the Internet. Last night a friend suggested I read “Hadoop Servers Expose Over 5 Petabytes of Data” which shows that Hadoop clusters are no different.
Guess what? Shodan was leveraged again, but this time to find Internet accessible Hadoop clusters. In aggregate it found clusters containing upwards of 5 Petabytes, which for those without a computer science degree that’s 5 million Gigabytes. The article goes on to mention that over the past year nearly 500 Hadoop systems have been held for ransom. The article then goes on to point out where to go to secure a Hadoop system. I bring all this up because very soon at Black Hat in July Solarflare will be demonstrating with Cloudwick how we can use the server NIC hardware to directly secure a Hadoop cluster. This can be done without changing a single line of code or altering the Hadoop configuration, stay tuned…