The Importance of “Local”

Binary translates to “Local”

We’ve all attended large industry international trade conferences hosting tens of thousands of people. These are spectacles designed to raise brand awareness, educate those in attendance about industry advances, network with colleagues you haven’t seen in a spell, all while promoting new products and services. By contrast there are also smaller regional industry trade shows that are scaled-down versions of these larger events with many of the same objectives, and then there are Security BSides events.

For those not familiar with BSides, they were started in 2009 to further educate folks on cybersecurity at the city and regional level. Think Blackhat, but on a Saturday at the local civic center, and with perhaps 200 people instead of 19,000. Let’s face it, most security engineers are introverts so socializing at significant events like Blackhat is uncomfortable. While bringing a few coworkers or friends on a Saturday to a BSides event can be downright fun. Let’s face who doesn’t want to sit for 20-30 minutes in the lock-pick village with their friends to test their skills on some of MasterLock, Schlage or Kwikset’s most common products. It’s heartwarming to teach a NOOB (short for a newbie) how to pick a lock, then watch their excitement when the hasp clicks open for the first time.

Then there’s always the Capture the Flag (CTF) or wireless CTF for when you’re not interested in the session(s) being offered. If you’ve not played a security capture the flag event before then you really are missing something. It is a challenging series of puzzles served up Jeopardy-style. Say 10 points if you can decrypt this phrase. Or 20 points if you can determine whose attacking your machine on five different ports. Perhaps another 50 points if you can write a piece of code that can read a web page, unscramble five words, and post the five proper words back to the website in three seconds before the clock expires and the words are no longer valid. It’s an intellectual problem solving competition at its finest, and did I mention there is a leaderboard. Often projected high on the wall for all to see throughout the day are the teams with the highest scores. It really warms the heart when your team is the second on the board and it stays in the top five most of the day. While we were the second on the board at BSides Asheville, we didn’t stay in the top five for long.

More seriously though, for a $20 entry fee (which includes a T-shirt) these BSides events offer an affordable local event for cybersecurity engineers and hobbyists. BSides enables socially challenged people the opportunity to step out of their shell, and reach out to similar like-minded individuals while networking in a comfortable and technical space. You can bond over lock-picking, a CTF challenge, during lunch or between sessions. Bring one of your nerd friends as a wingman, or better yet several to form a CTF team, and make a day of it. If you’d like to check out an online CTF one of our favorites is RingZer0. If you want to see the hacker side of the Technology Evangelist, W3bMind5, or read about his team’s experiences at BSides Asheville then they can be found at RedstoneCTF.

The RedstoneCTF team may be attending BSidesCLT on September 28th and BSidesRDU on October 19th.

East West​ Threat Made Real

Raspberry Pi 3B+ With Power over Ethernet Port in Plastic Case

Many in corporate America still don’t view East-West attacks as a real, let alone a significant threat. Over the past several years while meeting with corporate customers to discuss our future security product, it wasn’t uncommon to encounter the occasional Ostrich. These are the 38% of people who responded to the June 2018 SANS Institute report stating that they’ve not yet been the victim of a breach. In security we have a saying “There are only two types of companies, those that know they’ve been breached, and those that have yet to discover it.” While this sounds somewhat flippant, it’s a cold hard fact that thieves see themselves as the predators and they view your company as the prey. Much like a pride of female lions roaming the Africa savanna for a large herd, black-hat hackers go where the money is. If your company delivers value into a worldwide market, then rest assured there is someone out there looking to make an easy buck from the efforts of your company. It could be contractors hired by a competitor or nation-state actors looking to steal your product designs, a ransomware attacker seeking to extort money, or merely a freelancer surfing for financial records to access your corporate bank account. These threats are real, and if you take a close look at the network traffic attempting to enter your enterprise, you’ll see the barbarians at your gate.

A few months back my team had placed a test server on the Internet with a single “You shouldn’t be here” web page with a previously unused, unadvertised, network address. This server had all its network ports secured in hardware so that only port 80 traffic was permitted. No data of any value existed on the system, and it wasn’t networked back into our enterprise. Within one week we’d recorded over 48,000 attempts to compromise the server. Several even leveraged a family of web exploits I’d discovered and reported back in 1997 to the Lotus Notes Domino development team (it warmed my heart to see these in the logs). This specific IP address was assigned to our company by AT&T, but it doesn’t show up in any public external registry as belonging to our company, so there was no apparent value behind it, yet 48,000 attempts were made. So what’s the gizmo in the picture above?

In the January 2019 issue of “2600 Magazine, The Hacker Quarterly” a hacker with the handle “s0ke” wrote an article entitled “A Brief Tunneling Tutorial.” In it, s0ke describes how to set up a persistent SSH tunnel to a remote box under his control using a Raspberry Pi. This then enables the attacker to access the corporate network just as if he was sitting in the office. In many ways, this exploit is similar to sending someone a phishing email that then installs a Remote Access Trojan (RAT) on their laptop or desktop, but it’s even better as the device is always on and available. Yesterday I took this one step further. Knowing that most corporate networks leverage IP Phones for flexibility and that IP Phones require Power over Ethernet (PoE), I ordered a new Raspberry Pi accessory called a Pi PoE Switch Hat. This is a simple little board that snaps onto the top of the Pi and leverages the power found on the ethernet port to power the entire server. The whole computer shown above is about the size of a pack of cigarettes with a good sized matchbook attached. When this case arrives, I’ll utilize our 3D printer to make matching black panels that will then be superglued in place to cover all the exposed ports and even the red cable. The only physically exposed port will be a short black RJ45 cable designed to plug into a power over Ethernet port and two tiny holes so light from the power and signal LEDs can escape (a tiny patch of black electrical tape will cover these once deployed). 

When the Raspberry Pi software bundle is complete and functioning correctly, as outlined in s0ke’s article, then I’ll layer in accessing my remote box via The Onion Router (Tor) and pushing my SSH tunnel out through port 80 or 443. This should make it transparent to any enterprise detection tools. Tor should mask the address of my remote box from their logs. In case my Pi is discovered I’ll also install some countermeasures to wipe it clean when a local console is attached. At this point with IT’s approval, I may briefly test it in our office to confirm its working correctly. Then it becomes a show-and-tell box, with a single powerpoint slide outlining that east-west threats are real and that a determined hacker with $100 in hardware and less than one minute of unaccompanied access in their facility can own their network. The actual hardware may be too provocative to display, so I’ll lead with the slide. If someone calls me on it though I may pull the unit out of my bag and move the discussion from the hypothetical to real. If you think this might be a bit much, I’m always open to suggestions on better ways to drive a point home, so please share your thoughts.

Raspberry Pi 3B+ with Pi PoE Switch Hat

P.S. The build is underway, the Pi and Pi PoE Switch Hat have arrived. To keep the image as flexible as possible I’ve installed generic Raspbian on an 8GB Micro-SD card. Applied all updates, and have begun putting on custom code, system generically named “printer” at this point . Also, a Power over Ethernet injector was ordered so the system could be tested in a “production like” power environment. It should be completed by the end of the month, perhaps in time for testing in my hotel during my next trip. Updated: 2019-01-20

A persistent automated SSH tunnel has been set up between the “printer” and the “dropbox” system and I’ve logged into the “printer” by connecting via “ssh -p 9091 scott@localhost” on the “dropbox,” this is very cool. There is a flaw in the Pi PoE Switch board or its set up at this point as it is pulling the power off the ethernet port, but it is NOT switching the traffic so at this point the solution utilizes two Ethernet cables, one for power and the second for the signal. This will be resolved shortly. Updated: 2019-01-23

Raspberry Pi Zero on Index Finger

But why risk the Ethernet port not being a powered Ethernet jack, and also who wants to leave behind such a cool Raspberry Pi 3B+ platform behind when something with less horsepower could easily do the job? So shortly after the above intrusion device was functional I simply moved the Micro-SD card over to a Raspberry Pi Zero. A regular SD card is shown in the picture for the purpose of scale. The Pi Zero is awesome if you require a low power small system on a chip (SoC) platform. For those not familiar with the Pi Zero it’s a $5 single core 1Ghz ARM platform that consumes on average 100mw, so it can run for days on a USB battery. Add to that a $14 Ethernet to MicroUSB dongle and again you have a single cable hacking solution that only requires a generic Ethernet port. Of course it still needs a tight black case to keep it neat, but that’s what 3D printers are for.

Pi Zero, Ethernet Dongle
& USB Battery
(SD Card for Size Comparison)

Now, this solution will burn out in a couple of days, but as a hacker if you’ve not established a solid beachhead in that time then perhaps you should consider another line of work. Some might ask why I’m telling hackers how to do this, but frankly, they’ve known for years since SoC computers first became main stream. So IT managers beware, solutions like these are more common than you think, and they are leaking into pop culture through shows like Mr. Robot. This particular show has received high marks for technical excellence, and Myth Busters would have a hard time finding a flaw. One need only rewatch Season 1 episode 5, to see how a Raspberry Pi could be used to destroy tapes in a facility like Iron Mountain. Sounds unrealistic, then you must watch this Youtube video where they validate that this specific hack is in-fact plausible. The point is no network is safe from a determined hacker, from the CAN bus in your car, to building HVAC systems, or industrial air-gapped control networks. Strong security processes and policies, strict enforcement, and honeypot detection inside the enterprise are all methods to thwart and detect skilled hackers. Updated: 2019-01-27

TE18: Personal Cyber Security

Today we do something different and Bob Van Valzeh and I have a frank discussion on personal security in a digital world.

Below are some of the topics we discuss:

  • Background on Scott’s hacking past and his conversion to a white hat, a friendly, hacker.
  • Personal security, and acting smartly.
  • Don’t carelessly create opportunities for hackers to exploit your weaknesses.
  • The importance of password managers.
  • Being aware of threats around you social engineering, skimmers, etc…
  • Banking today, avoid using checks, online banking.
  • Security of free wifi, and the man in the middle
  • Being careful with email and phone usage.
  • Updating your Internet of Things (IoT) devices, thermostats, routers, personal devices, etc…

We then wrap it up with a quick rant on the government hoarding cybersecurity exploits.

TE17: The Low Down on The Meltdown

Yesterday Brantley Coile, CEO of Coraid and the original developer of NAT (Network Address Translation) joined me to discuss the Intel Meltdown vulnerability, and how we got here. Last Friday Brantley authored a Linkedin Blog entry titled “Intel Flubs Again.”

Below are some of the topics we discussed for this podcast:

  • How we met over a decade ago, and the Plan9 OS, not the movie.
  • Network Address Translation, and how he and a partner invented it, Private Internet eXchange (Cisco PIX)
  • Brantley invented the first: stateful inspection firewall, VPN, and load balancer (Cisco Director).
  • How we’re stuck with Intel’s complex processor architecture.
  • Complex Instruction Set Computing (CISC) versus Reduced Instruction Set Computing (RISC).
  • The graveyard of architectures MIPS, Itanium, and i960 (we didn’t mention SPARC, PA, Alpha).
  • The evolution from the IBM 360/370 to the IBM 801 (ROMP), Power, and how RISC and CISC meet up.
  • How compilers hide architecture from developers.
  • Complex architectures inevitably lead to high likelihoods for vulnerabilities.
  • A simplified description of what Meltdown really is.
  • Pipelining and speculative execution.
  • The Low Down on the Meltdown, and how it exposes memory during the speculative phase, and how it recovers the contents of that memory post speculation.
  • What can Intel do, and how will it hurt performance?
  • This creates an opportunity for ARM.
  • What Mom needs to do to protect herself today from Meltdown.
  • and more…

Interested in learning more about Solarflare’s Meltdown Prevention Program?
Please send an email to sschweitzer@solarflare.com

TE11: A Firewall in the NIC

Tonight we had a discussion with Steve Pope, Solarflare’s CTO and Founder, on Solarflare’s new “Firewall in the NIC” capability called ServerLock that goes into Beta the end of December.

During our time together we reviewed the following:

  • What exactly does Solarflare mean when they say they’ve put a firewall in the NIC?
  • How does this improve the security of my server?
  • What is micro-segmentation, and how can this be applied to my applications, containers or VMs?
  • Why is having a firewall in the NIC better than a software firewall which is part of the OS?
  • Why is a firewall in the NIC better than say a top of rack firewall?
  • How much might this cost me in latency if the NIC is filtering every packet?
  • Who has Solarflare built this product for?
  • Where in my enterprise infrastructure should I consider using such a NIC?
  • Can this be used as an edge solution to enhance the security of my customer facing web servers, possibly further protecting them from a DDoS attack?
  • Where does Solarflare go from here, what’s next?

Interested in evaluating Solarflare’s ServerLock Firewall in the NIC technology?
Please send an email to sschweitzer@solarflare.com

TE1: Hadoop and Security

On June 8th, 2017 Ron Miller of Cloudwick joined us to go deep on Hadoop and Securing Hadoop Clusters. Ron is a member of the CTO staff at Cloudwick, and he’s been an industry leader in security for over two decades. I was fortunate enough to work with Ron for several years while he was with Solarflare, and we’ve remained friends since his departure. In 2015 we had the opportunity to work Black Hat and attend DEFCON together, and it was amazing to hear his perspective on the security products and events.

Four Container Networking Benefits

ContainerContainer networking is walking in the footsteps taken by virtualization over a decade ago. Still, networking is a non-trivial task as there are both underlay and overlay networks one needs to consider. Underlay Networks like a bridge, MACVLAN and IPVLAN are designed to map physical ports on the server to containers with as little overhead as possible. Conversely, there are also Overlay networks that require packet level encapsulation using technologies like VXLAN and NVGRE to accomplish the same goals.  Anytime network packets have to flow through hypervisors or layers of virtualization performance will suffer. Towards that end, Solarflare is now providing the following four benefits for those leveraging containers.

  1. NGINX Plus running in a container can now utilize ScaleOut Onload. In doing so NGINX Plus will achieve 40% improvement in performance over using standard host networking. With the introduction of Universal Kernel Bypass (UKB) Solarflare is now including for FREE both DPDK and ScaleOut Onload for all their base 8000 series adapters. This means that people wanting to improve application performance should seriously consider testing ScaleOut Onload.
  2. For those looking to leverage orchestration platforms like Kubernetes, Solarflare has provided the kernel organization with an Advanced Receive Flow Steering driver. This new driver improves performance in all the above-mentioned underlay networking configurations by ensuring that packets destined for containers are quickly and efficiently delivered to that container.
  3. At the end of July during the Black Hat Cyber Security conference, Solarflare will demonstrate a new security solution. This solution will secure all traffic to and from containers with enterprise unique IP addresses via hardware firewall in the NIC.
  4. Early this fall, as part of Solarflare’s Container Initiative they will be delivering an updated version of ScaleOut Onload that leverages MACVLANs and supports multiple network namespaces. This version should further improve both performance and security.

To learn more about all the above, and to also gain NGINX, Red Hat & Penguin Computing’s perspectives on containers please consider attending Contain NY next Tuesday on Wall St. You can click here to learn more.

Beyond SDN: Micro-Segmentation, Macro-Segmentation or Application-Segmentation Part-2

Large publicly traded companies like Cisco, EMC (VMWare) and Arista Networks are deeply entrenched with their customers giving them a beachhead on which they can fairly easily launch new products. Since their brands, and value is well understood and established it’s often a matter of just showing up with a product that is good enough to win new business. By contrast, start-ups like Illumio and Tufin have to struggle to gain brand recognition and work exceptionally hard to secure each and every proof of concept (PoC) engagement. For a PoC to be considered successful these new startups have to demonstrate significant value to the entrenched players as they also need to overcome the institutional inertia behind every buying decision.  So how are Illumio or Tufin any different, and what value could they possibly deliver to justify even considering them? While both Illumio and Tufin are focused on making enterprises and the deployment of enterprise applications more secure, they each leverage a dramatically different approach. First, we’ll explore Tufin, then Illumio.

Tufin has a feature called the Interactive Topology Map, which enables them to traverse your entire physical network, including your use of hybrid clouds to craft a complete map of your infrastructure. This enables them to quickly display on a single pane of glass how everything in your enterprise is connected. They then offer visual path analysis from which you can explore your security and firewall policies across your network. Furthermore, you can use a sophisticated discovery mechanism by which you select two systems, and it explores the path between them and displays all the security policies that might impact data flows between these two systems. In actual practice, as you define an application within Tufin you can leverage this sophisticated discovery or manually define the source, destination, and service. Tufin will then show you the status of the connection, at which point you can drill down to see what if any components in your infrastructure require a change request. They then have a six-step change ticket workflow: Request, Business Approval, Risk Identification, Risk Review, Technical Design, and Auto Verification. To date they appear to support the following vendors: Cisco, Check Point, Palo Alto Networks, Fortinet, Juniper, F5, Intel Security, VMWare NSX, Amazon Web Services, Microsoft Azure, and OpenStack.

By contrast, Illumio takes a much different approach, it designs security from the inside out with no dependencies on infrastructure. They attach an agent to each enterprise application as it is launched. This attached agent then watches over every network flow into and out of the application and records exactly what the application requires to be effective. From this, it computes a security policy for that application that can then be enforced every time that application is launched. It can then adapt to workflow changes, and it also has the capability to encrypt all data flowing between systems. While their videos and data sheets don’t specifically say this it sounds as though they’ve placed a shim into the network OS stack hosting the application so that they can then record all the network traffic flow characteristics, that’s likely how they support on the fly encryption between nodes. They do call out that they use IPTables, so it is possible that their code is an extension of this pre-existing security platform. Clearly, though they are just above the adapter, and Jimmy Ray confirms this in one of his awesome videos that Illumio is based on an “adapter security platform” view. Illumio then provides an enterprise management application to gather the flow data from all these agents to craft, and manage its view of the network.

So while Tufin looks into your network from the outside and enumerates what it finds, Illumio looks from the application out. Both are impressive and yield interesting perspectives worthy of evaluating. Moving forward both are tuned to delivery Application-Segmentation, it will be interesting to see how the market evaluates each, both had strong presences at RSA2016, but it will ultimately be revenue from customers that determines success.