10G/40G NIC Partitioning Using SR-IOV or PF-IOV Modes

scottcschweitzer networking

Partitioning a network interface card (NIC) so that multiple virtual machines (VMs) can use it at the same time has always been challenging. Last week Solarflare released an updated network device driver for Linux that now supports Single Route Input Output Virtualization (SR-IOV) and Physical Function Input Output Virtualization (PF-IOV) modes.  The actual details for setting up the adapter to leverage these modes can be found in the Solarflare Adapter User Guide.

Borrowing from the User Guide:

SR‐IOV enabled on Solarflare adapters provides accelerated cut‐through performance and is fully compatible with hypervisor based services and management tools. The advanced design of the Solarflare SFN7000 series adapter incorporates a number of features to support SR‐IOV. These features can be summarized as follows:

Multiple PCIe Physical Functions (PF).

Each physical port on the dual‐port 10G or 40G adapter can be exposed to the OS as multiple physical functions. A total of 16 PFs are supported per adapter with each having a unique MAC address.

PCIe Virtual Functions (VF).

A PF can support a configurable number of PCIe virtual functions. In total 240 VFs can be allocated between the PFs. The adapter can also support a total of 2048 MSI‐X interrupts.

Layer 2 Switching Capability.

A layer 2 switch configured in firmware supports the transport of network packets between PCI physical functions (PF) and Virtual functions (VF). This allows received packets to be replicated across multiple PFs/VFs and allows packets transmitted from one PF to be received on another PF or VF.

  • On a 10GbE dual‐port adapter each physical port can be exposed as a maximum 8 PFs
  •  On a 40GbE dual‐port adapter (in 2*40G mode) each physical port can be exposed as a maximum 8 PFs. 
  • On a 40GbE dual‐port adapter (in 4*10G mode) each physical port can be exposed as a maximum 4 PFs. 

All of this allows each VM to receive it’s own Virtual NIC. This enables applications to communicate directly with the wire via Hypervisor Bypass, PCI Passthru or DirectPath I/O.  This restores native network performance to VMs. So with dual port 10G adapters, Solarflare provides support for 8PFs per physical port and a total of 240 VFs per adapter.

Furthermore, Solarflare also supports Kernel Bypass alongside Hypervisor Bypass to deliver new bare metal ultra-low latency from within a VM.

To learn more drop me an email.

Leave a Reply