Security

1st Ever Firewall in a NIC

scottcschweitzer Microsegmentation, networking, Security

Last week at Black Hat Solarflare issued a press release debuting their SolarSecure solution, which places a firewall directly in your server’s Network Interface Card (NIC). This NIC based firewall called ServerLock, not only provides security, but it also offers agentless server-based network visibility. This visibility enables you to see all the applications running on every ServerLock enabled server. You can then quickly and easily develop security policies which can be used for compliance or enforcement. During the Black Hat show setup, we took a 10-minute break to have an on-camera interview with Security Guy Radio that covered some of the key aspects of SolarSecure.

SolarSecure has several very unique features not found in any other solution:

  • Security and visibility are entirely handled by the NIC hardware and firmware, there are NO server side software agents, and as such, the solution is entirely OS independent.
  • Once the NIC is bound to the centralized manager it begins reporting traffic flows to the manager which then represents those graphically for the admins to easily turn into security policies. Policies can be created for specific applications, enabling application level network segmentation.
  • Every NIC maintains separate firewall tables for each local IP address hosted on the NIC to avoid potential conflicts from multiple VMs or Containers sharing the same NIC.
  • Each NIC is capable of handling over 5,000 filter table rules along with another 1,000 packet counters that can be attached to rules.
  • Packets transit the rules engine between 50 and 250 nanoseconds so the latency hit is negligible.
  • The NIC filters both inbound and outbound packets. Packets which are dropped as a result of a match to a firewall rule generate an alert on the management console and inbound packets consume ZERO host CPU cycles.

Here is a brief animated explainer video which was produced prior to the show that sets up the problem and explains Solarflare’s solution. We also produced a one-minute demonstration of the management application and its capabilities.

What’s a Smart NIC?

scottcschweitzer Microsegmentation, networking, Security

While reading these words, it’s not just your brain doing the processing required to make this feat possible. We’ve all seen over and under exposed photos and can appreciate the decision making necessary to achieve a perfect light balanced photo. In the laboratory, we observed that the optic nerve connecting the eye to the brain is responsible for measuring the intensity of the light hitting the back of your eye. In response to this data, each optic nerve dynamically adjusts the aperture of the iris in your eye connected to this nerve to optimize these levels. For those with some photography experience, you might recall that there is a direct relationship between aperture (f-stop) and focal length. It also turns out that your optic nerve, after years of training as a child, has come to realize you’re reading text up close, so it is now also responsible for modifying the muscles around that eye to sharpen your focus on this text. All this data processing is completed before your brain has even registered the first word in the title. Imagine if your brain was responsible for processing all the data and actions that are required for your body to function properly?

Much like your optic nerve, the difference between a standard Network Interface Card (NIC) and a smart NIC is how much processing the Smart NIC offloads from the host CPU. Until recently Smart NICs were designed around Field Programmable Gate Array (FPGA) platforms costing thousands of dollars. As their name implies, FPGAs are designed to accept localized programming that can be easily updated once installed. Now a new breed of Smart NIC is emerging that while it isn’t nearly as flexible as an FPGA, they contain several sophisticated capabilities not previously found in NICs costing only a few hundred dollars. These new affordable Smart NICs can include a firewall for security, a layer 2/3 switch for traffic steering, several performance acceleration techniques, and network visibility with possibly remote management.

The firewall mentioned above filters all network packets against a table built specifically for each local Internet Protocol (IP) address under control. An application processing network traffic is required to register a numerical network port. This port then becomes the internal address to send and receive network traffic. Filtering at the application level then becomes a simple process of only permitting traffic for specific numeric network ports. The industry has labeled this “application network segmentation,” and in this instance, it is done entirely in the NIC. So How does this assist the host x86 CPU? It turns out that by the point at which operating system software filtering kicks in the host CPU has often expended over 10K CPU cycles to process a packet. If the packet is dropped the cost of that drop is 10K lost host CPU cycles. If that filtering was done in the NIC, and the packet was then dropped there would be NO host CPU impact.

Smart NICs also often have an internal switch which is used to steer packets within the server rapidly. This steering enables the NIC to move packets to and from interfaces and virtual NIC buffers which can be mapped to applications, virtual machines or containers. Efficiently steering packets is another offload method that can dramatically reduce host CPU overhead.

Improving overall server performance, often through kernel bypass, has been the providence of High-Performance Computing (HPC) for decades. Now it’s available for generic Ethernet and can be applied to existing and off the shelf applications. As an example, Solarflare has labeled its family of Kernel Bypass accelerations techniques Universal Kernel Bypass (UKB). There are two classes of traffic to accelerate: network packet and application sockets based. To speed up network packets UKB includes an implementation of the Data Plane Development Kit (DPDK) and EtherFabric VirtualInterface (EF_VI), both are designed to deliver high volumes of packets, well into the 10s of millions per second, to applications familiar with these Application Programming Interfaces (APIs). For more standard off-the-shelf applications there are several sockets based acceleration libraries included with UKB: ScaleOut Onload, Onload, and TCPDirect. While ScaleOut Onload (SOO) is free and comes with all Solarflare 8000 series NICs, Onload (OOL) and TCPDirect require an additional license as they provide micro-second and sub-microsecond 1/2 round trip network latencies. By comparison, SOO delivers 2-3 microsecond latency, but the real value proposition of SOO is the dramatic reduction in host CPU resources required to move network data. SOO is classified as “zero-copy” because network data is copied once directly into or out of your application’s buffer. SOO saves the host CPU thousands of instructions, multiple memory copies, and one or more CPU context switches, all dramatically improve application performance, often 2-3X, depending on how network intense an application is.

Finally, Smart NICs can also securely report NIC network traffic flows, and packet counts off the NIC to a centralized controller. This controller can then graphically display for network administrators everything that is going on within every server under its management. This is real enterprise visibility, and since only flow metadata and packet counts are being shipped off NIC over a secure TLS link the impact on the enterprise network is negligible. Imagine all the NICs in all your servers reporting in their traffic flows, and allowing you to manage and secure those streams in real time, with ZERO host CPU impact. That’s one Smart NIC!

5 Petabytes of Public Hadoop Data

scottcschweitzer Security ,

HadoopSecurityBack in 2015, I sat through a DOE presentation during a government cyber security conference on SCADA (Supervisory Control and Data Acquisition) systems accessible from the web. SCADA is used to allow computers to manage public utilities, water, gas, petroleum refineries, nuclear power plants, etc… The speaker did a live demo using Shodan where he was able to demonstrate something like over 65K open SCADA networks reachable from the Internet. This article backs up the above-mentioned presentation, though the author points out that the maps only show German made SCADA systems. To be more precise the maps show Seimens SCADA controllers, which dominate the market. Most of these systems were for industrial control, and they should have been air-gapped, physically not connected, to ANY external network, let alone the Internet. Last night a friend suggested I read “Hadoop Servers Expose Over 5 Petabytes of Data” which shows that Hadoop clusters are no different.

Guess what? Shodan was leveraged again, but this time to find Internet accessible Hadoop clusters. In aggregate it found clusters containing upwards of 5 Petabytes, which for those without a computer science degree that’s 5 million Gigabytes. The article goes on to mention that over the past year nearly 500 Hadoop systems have been held for ransom. The article then goes on to point out where to go to secure a Hadoop system.  I bring all this up because very soon at Black Hat in July Solarflare will be demonstrating with Cloudwick how we can use the server NIC hardware to directly secure a Hadoop cluster. This can be done without changing a single line of code or altering the Hadoop configuration, stay tuned…

Black Hat 2016: NIC Packet Filtering

scottcschweitzer Security

Solarflare wants to talk with you at Black Hat in Las Vegas next month, and we’re raffling off a Wifi Pineapple to those who sign up for a meeting. What is a Wifi Pineapple you ask, perhaps one of the best tools available for diagnosing wireless security issues?

At Black Hat, Solarflare will be talking about their new line of SFN8xxx series adapters that support five-tuple packet filtering directly in hardware. The SFN8xxx series adapters support thousands of filters and an additional one thousand counters that can be applied to track filter usage. Along with filtering, we’ll be discussing the tamper-proof nature of this new line of adapters, and its capability to support over the wire firmware or filter table updates via an SSL/TLS link directly to the controller on the adapter.

To learn more or set up a meeting for Wednesday, August 3 or Thursday, August 4th at Black Hat please send an email to scollins@solarflare.com, and you’ll be automatically enrolled in our drawing for a Wifi Pineapple.

Popular Cyber Security & Networking Acronyms

scottcschweitzer networking, Security

Thanks in part to IBM, no field uses acronyms like computer science. Recently I was asked to pick up product management responsibility for an evolving new line of cybersecurity products. To be relevant one needs to be current with the trendy lingo, as such here are some of the must-know acronyms. So if you’re a budding cyber security student, or hacker in training see how many of these you know off the top of your head. Note, every one of these has been checked, and the full expansion of the acronym links to the appropriate Wikipedia page.  Good luck, and enjoy.

ACL – Access Control List
AES – Advanced Encryption Standard
APT – Advanced Persistent Threat
ARP – Address Resolution Protocol
AV – AntiVirus
BGP – Border Gateway Protocol
BIND – Berkley Internet Name Domain
CAL – Client Access License
CERT – Computer Emergency Response Team
CMDB – Configuration Management Database
CRC – Cyclic Redundancy Check
CVE – Common Vulnerabilities and Exposures
DES – Data Encryption Standard
DLP – Data Loss Prevention
DNS – Domain Name System, although Service is also common
EAP – Extensible Authentication Protocol
EGP – Exterior Gateway Protocol
EMM – Enterprise Mobility Management
HIPS – Host-based Intrusion Prevention System, Wikipedia doesn’t have a unique listing for this one.
IPS – Intrusion Prevention System
IPsec – Internet Protocol Security
IDS – Intrusion Detection System
L2F – Layer-2 Forwarding protocol
L2TP – Layer-2 Tunneling Protocol
LDAP – Lightweight Directory Access Protocol
MAC – Media Access Control, as in MAC address
MAC – Mandatory Access Control
NAC – Network Access Control
NAT – Network Address Translation
NGFW – Next-Generation Firewall
NIST – National Institute of Standards and Technology
OSI – Open Systems Interconnection
OSPF – Open Shortest Path First
PAP – Password Authentication Protocol
PFS – Public-key Forward Secrecy
PGP – Pretty Good Privacy
PKI – Public Key Infrastructure
PPP – Point to Point Protocol
PPTP – Point to Point Tunneling Protocol
RARP – Reverse Address Resolution Protocol
RIP – Routing Information Protocol
RSA – Rivest-Shamir-Adleman
SCAP – Security Content Automation Protocol
SDN – Software Defined Networking
SIEM – Security Information Event Management
SNMP – Simple Network Management Protocol
SSH – Secure Shell
SSL – Secure Sockets Layer
TLS – Transport Layer Security
URI – Uniform Resource Identifier
VPN – Virtual Private Network
WAP – Wireless Application Protocol
WEP – Wired Equivalent Privacy

If it’s not on this list that doesn’t mean it’s not important, just that I’ve not yet stumbled across it, or Wikipedia doesn’t recognize it as an industry term but is likely a vendor-initiated one.

Next Generation Network (NGN) Forensics

scottcschweitzer Capture, networking, Security

In two weeks I’ll be hosting two events: one near the NSA at the Courtyard Fort Meade on the 27th, and the second a few miles from the CIA at the Hilton Mclean Tyson Corner. Both will start at 4 PM, have a considerably strong Network Forensics agenda, and then wrap up by 7 PM. Both events are identical, they’re free to attend, and we’ll be providing beer, wine & soda along with appetizers.

Below is the current agenda, but it’s still somewhat fluid at this point as it’s the first NGN we’re doing that is focused on Network Forensics, and is setup for the Intelligence Community. The dozen or so prior NGNs, and another later that week have been centered on the Financial Trading market where our leadership in low latency is well known. Here is the current agenda:

  • 3:30 – 4:00  Registration, drinks and snacks
  • 4:00 – 4:05  Welcome & Introductions – Paul Bravo, Solarflare VP US/OEM Sales
  • 4:05 – 4:35  Keynote Speaker – David Riddoch, Founder, Solarflare
  • 4:35 – 4:50  Dell – Cameron Chehreh, CTO Federal
  • 4:50 – 5:35  Panel: Deep Packet Inspection (Snort, Suricata & Bro)
  • 5:35 – 5:45  Reservoir Labs – Alison Ryan, VP Business Development
  • 5:45 – 6:15  Advances in Capture – Ron Miller, Director Technical Marketing, Solarflare
  • 6:15 – 6:45  Open Q&A of all speakers
  • 6:45 – 7:30  Reception & Demonstrations

I’ll moderate the panel on Deep Packet Inspection which will include:  Eileen Donnelly, US Government, Erik Mogus Director Sales Engineering for Reservoir Labs, and someone to be named shortly. The purpose of the panel is to compare and contrast the different approaches to packet inspection so someone familiar with one can see different perspectives on the other two.  The audience will also be encouraged to ask questions of the panelists.

To sign up for either event please use one of the links below:

Tuesday, October 27th l Time: 3:30-7:30 PM
Courtyard Marriott Fort Meade, 2700 Hercules Rd, Annapolis Junction, MD
Click to attend
Wednesday, October 28th l Time: 3:30-7:30 PM
Hilton McLean Tysons Corner, 7920 Jones Branch Drive, McLean, VA
Click to attend

 

Detecting Data Breaches in Real Time

scottcschweitzer Security

Not a week passes without news of another company announcing a data security breach. Many of these breaches start with the Point of Sale (POS) systems, but as we saw with Anthem, Sony and Edward Snowden, that isn’t always the case. Regardless of where the breach starts, nearly all of the valuable data lost flows through, and eventually out of the enterprise. Imagine if a small team of clowns walked into your business in the middle of the day, went straight to your server room, pulled out big clown scissors, cut all the cables front and back on your servers and proceeded to carry them out to their clown car. Certainly, employees would question what was going on, and surely someone would stop them before the servers actually left the building. Today that’s exactly what’s happening; only the clowns are black hat hackers acting remotely.

All companies have firewalls, many have intrusion detection systems, and some install intrusion prevention systems, but does your company capture and analyze all the traffic flows entering, and leaving your enterprise? Even more daunting, imagine capturing all of the flows within your company, then scrubbing that data looking for unique traffic patterns, perhaps in real time? At then end of December Norse specifically identified the Sony employee who was laid off in May, and who departed with tens of gigabytes of Sony movies and digital assets. This employee was someone in IT, possibly very much like you who had access to many of the digital security certificates, admin ids and passwords within Sony, many of those items were included in files and spreadsheets that Gods Of Peace released. Sony knew months before that they were separating people from the business, had they been looking for unusual internal network traffic patterns they might very well have been able to thwart this digital theft.

For the rest of the story please visit the full article in this months issue of Cyber Defense Magazine.

A 10GbE Capture Platform: Snort, Bro, Suricata & Wireshark

scottcschweitzer networking, Security, Time

Perhaps you’re responsible for your companies network security, or maybe you’re designing an appliance for your business?  If so, then you’ve likely already become familiar with Snort, Bro,  Suricata, and Wireshark. As you may have recently discovered for real performance with these applications at 10GbE speeds you need the proper adapter, and capture driver or you risk dropping vast numbers of packets. Furthermore, it is now possible to not only capture a copy of the packets received on the server but also transmitted. All while running your programs on other cores within the same server. Furthermore, if you’re interested in capturing all the received, transmitted, and virtual machine (VM) to VM traffic within your server then you can actually designate one VM to capture a copy of all the network traffic for analysis. To further sweeten things, this can also be done from within a Docker container built to handle capture.

Some might ask why you would want to capture transmitted packets and run them through Snort, Bro or Suricata? Simple, to look for outbound traffic patterns that might indicate a breach. Perhaps a VM on one of your servers has been compromised, and it is sending out your companies precious AutoCAD files in the middle of the night to a country in Asia you don’t do business with. If you’re not looking at transmitted packets you may never detect, or stop a breach of this nature. Setting up rules to look for file transfers of specific types, during specific times, or conforming to other criteria specific to outbound traffic is a fairly new trend. Also, this capture doesn’t have to be packets on your server, you can take a more traditional approach and dedicate a server for capture in every rack, then use an optical tap or the spanning port off a switch. In fact, you can install multiple adapters and aggregate the ports together until you hit the performance limits of your system.

Most accelerated 10G capture platforms require both a performance adapter and a special purpose capture driver.  Furthermore to capture both received and transmitted packets in parallel you have only one choice, and that is an adapter and software from Solarflare. You can start with Solarflare’s Flareon Ultra SFN7122F adapter and a SolarCapture Live license, and as your needs grow scale to their dual 40G adapter the SFN7142Q.

Solarflare provides this high-performance capture platform designed specifically for engineers looking to build leading edge security solutions. Let’s take a closer look at the adapter and software. The network server adapter, the Solarflare SFN7122F, is a board that contains one Solarflare single core Ethernet controller chip. This Ethernet controller core on this chip has multiple packet engines each dedicated to processing received or transmitted packets. This enables the SFN7122F adapter to support wire-rate lossless packet capture, even with huge bursts of the smallest sized packets (64 bytes each) on a single port. This dedication of resources enables transmitting wire-rate 64-byte packets at the same time, on the same interface, and in parallel, without impacting capture performance. Furthermore, the SFN7142Q utilizes the same Ethernet controller, but with two of these on the same chip so it can support capture on two 40G ports, or four 10G port, or wire-rate lossless capture on two 10G ports.

The next component in this platform is SolarCapture Live (SCL), which provides a complete Libpcap replacement library, and a Snort DAQ interface. This allows for two fairly seamless methods for easily connecting to Snort. If SCL is initialized in cluster mode it can spawn multiple capture instances, up to one per core, and deliver all network packets in Libpcap format spread across these cores. SCL then uses advanced receive flow steering to flow-hash the packets across all of these capture nodes within the capture cluster. Flow-hashing is the process of looking at several key fields in the packet header then always routing all the traffic from a given flow consistently to the same cluster node (core) so security applications like Snort, Suricata and Bro can always see all the given data for that specific network flow.

This Solarflare capture platform also supports an optional Solarflare Precision Time Protocol (PTP) software license that can accept an external hardware Pulse Per Second (PPS) signal (via an additional optional bracket kit) which provides the necessary mini-BNC connectors that can then be used to attach the adapter to an external master clock. Unlike similar adapters, this optional PCIe faceplate has a second mini-BNC connector to support daisy chaining the clock signal out of the adapter into another adapter. These Solarflare adapters include a highly precise clock chip, the Stratum 3, this ensures that time stamping is accurate to within 100 nanoseconds from the PTP master, precision time stamping is typically only available on much more expensive FPGA based adapters. Furthermore, the PTP license enables time stamping for the capture of both received and transmitted packets, so you can use it to measure application performance. Additionally, Solarflare’s 100 nanosecond precision is 15X more precise than a competing adapter at a similar price point that only captures and time stamps inbound packets.

So if you’re looking to get into packet capture for security monitoring or performance analysis, please consider contacting Solarflare, and ask about their SFN7122F with SolarCapture Live. You’ll be pleasantly surprised at how well it performs when compared to the much more expensive FPGA based solutions which sell for 5X or more the price of this unique bundle.

A First Generation Firewall in Your NIC

scottcschweitzer networking, Security

Earlier this year Solarflare released a software driver for their line of Flareon adapters called SolarSecure. SolarSecure is a stateless packet filtering engine with a rich set of features that maps almost perfectly into what Wikipedia has defined as a first generation packet filtering firewall. This is meant to separate it from the more common second generation state-full firewall & third generation application level firewalls that have become the more commonly accepted definition of a firewall today.

Earlier this morning someone asked me how SolarSecure might help a huge cloud service provider, say one embroiled in a celebrity scandal.  To fully leverage SolarSecure to make production Internet servers more secure there are at least five potential use cases, and they are:

1. SolarSecure could be used to augment existing firewalls by providing a high bandwidth front end method for rate limiting inbound traffic from chatty address, ex. no more than 10 packets in 10 milliseconds from any specific source (anything over that will be dropped) while also filtering by TCP address. Here is a sample configuration file that one could use to do just this.

2. On all Internet facing interfaces, one should also check the TCP flags to thwart a SYN flood attack. SYN floods make up just over 30% of the attack traffic every day. Another configuration example can be found here that addresses this use case.

3. Also for all Internet facing ethernet interfaces, one should use SolarSecure in conjunction with Norse’s Darklist, or a similar provider, to do blacklist filtering to ensure that known bad actors on the Internet can’t impact my systems at all by dropping their traffic immediately. Again here is some code demonstrating IP address filtering.

4. As a method within the DMZ to tightly control what every server port can be used for.  For example, Internet facing ports should only pass traffic for permitted UDP/TCP ports that are specific to the service that system offers, if it is a web server then white list the system to deny (drop all traffic) on that port that is NOT port 80.  Also for backend service interfaces on that same system white list both the addresses & ports permitted on each & every ethernet interface to ensure that sideways attacks from other DMZ systems that might have been compromised is locked out. Here is an example of how to drop SSH requests to an interface.

5. Finally, HTTP request filtering. Looking at the actual requests coming into the server & only accepting ones that meet the objectives of the server itself.  Hackers will attempt to get in via the web server interface, if SolarSecure is only passing the web server specific types of pre-approved requests then the web server is much less exposed. One final example is here.

If you have any additional questions or comments, don’t hesitate to reach out to me.

A 10G Bouncer for your Network Server Ports

scottcschweitzer networking, Security

We’ve all been to an exclusive club where we may have been denied entry because we weren’t on “the list.”  Does your server have a “list?” It should.

Friday the Department of Homeland Security informed us that over 1,000 US business have been hit by a new hacker tool called “Backoff.” UPS it seems got the most negative press on this one, and “Backoff” had ONLY compromised 1% of their US stores, 51 in total. Reporting the cyber break in made nationwide news on Friday.  So this week UPS is the new face of cyber victims, for the time being replacing Target who suffered an estimated $2.5B loss due to a similar incident last winter.

Unlike Target, which started with weak security on an HVAC system, UPS was hit by a very sophisticated piece of software called “Backoff.” This hacker tool contains a key logger, memory scraping (yes it crawls through memory looking for userids, passwords, system names, credit card info, etc…) and it plays several cute tricks to immunize itself from removal. So how do you minimize & possibly eliminate your servers from being hit by “Backoff?” One method would be to install a white list based security firewall in your server, essentially putting a bouncer on your server’s network port.

This network server bouncer would then only let systems enter your server that come in from places on the list, and they would only be allowed in via communications ports they were pre-approved for. In other words, say you have a web server on an internal address of 10.0.0.77 and you’ve configured it to talk to your database server on port 7940.  Your database server also has this network server bouncer running, and he’s using a whitelist that says: allow 10.0.0.77:7940.  Now suppose your web server was hacked the only possible way the attacker could jump from the web server to the database server would be through port 7940.  All the other ports: telnet, ssh, ftp, etc… from 10.0.0.77 don’t exist, those paths from the web server to the database server are unavailable to the attacker because they aren’t on the whitelist. To actually get into the database server the attacker would have to communicate as if he were a database server himself because that is all the database server will permit on port 7940.  Also remember that HVAC server that brought down Target, it wouldn’t be on the list at all to access any of your core business servers so the sideways attack Target experienced would be impossible.  Your data, on these servers at least, would be considerably more secure.

Taking this one step further, as the admin you could then use this whitelist capability on each ethernet server port to logically wire together your infrastructure such that only systems, and admin workstations that should talk with each other on predetermined, acceptable, ports can. A typical web server might have a whitelist that contains a pool of database servers, on a fixed set of ports, a management server or two, also talking on a different set of permitted ports, and a management workstation or few that also can only communicate over a very narrow band of ports. This would all be enforced in the network server adapter’s hardware that was executing the white list, being the bouncer, so ANY attempted access is stopped before it reaches the operating system.

Does such a magical network server bouncer with a white list on his clipboard exist today, yes.  It’s SolarSecure running on Solarflare’s Flareon network server adapters. With it you can specify a list of acceptable individual IP addresses or ranges of IP addresses that can communicate with a server on a given ethernet interface.  You can then filter off all communications on unapproved ports, in a second layer of filtering.  To learn more post something here, or drop me an email.