Switch Embedded Teaming

Switch Embedded Teaming (SET) is a new feature in the Software-Defined Networking stack that will be included in Windows Server 2016. It enables to group several physical network adapters (from one to eight) into a single virtual network adapters in a Hyper-V environment. In fact, it is SET is an alternative to standard NIC teaming.

The main advantage of SET compared to standard NIC teaming is the RDMA convergence and the RDMA virtualization for host. In this way storage traffic with SMB direct can be converged with others traffics. Below you can find a Microsoft diagram which compares the network convergence in Windows Server 2012 R2 and Windows Server 2016:

As you can see above, in Windows Server 2016, you’ll need less Physical NICs for the same job! Obviously at least 10GB NICs and QoS management will be recommended in production.

Network technologies supported and unsupported

Switch Embedded Teaming is compatible with the following network technologies:

• Datacenter bridging (DCB)
• Hyper-V Network Virtualization – NV-GRE and VxLAN are both supported in Windows Server 2016 Technical Preview.
• Receive-side Checksum offloads (IPv4, IPv6, TCP) – These are supported if any of the SET team members support them.
• Remote Direct Memory Access (RDMA)
• SDN Quality of Service (QoS)
• Transmit-side Checksum offloads (IPv4, IPv6, TCP) – These are supported if all of the SET team members support them.
• Virtual Machine Queues (VMQ)
• Virtual Receive Side Scalaing (RSS)

However SET is not compatible with these network technologies:

• 802.1X authentication
• IPsec Task Offload (IPsecTO)
• QoS in host or native OSs
• Receive side coalescing (RSC)
• Receive side scaling (RSS)
• Single root I/O virtualization (SR-IOV)
• TCP Chimney Offload
• Virtual Machine QoS (VM-QoS)

Physical NICs requirements

First the physical NICs that will be in the SET must pass the Windows Hardware Qualification and Logo. Secondly, each physical NICs must be the same (same manufacturer, model firmware and driver). To finish you can group from one to eight physical NICs in the same team.

SET settings

First of all, when you use SET, all physical network adapters are in active mode. You can’t implement a physical NIC in standby mode in the team.

Next the only teaming mode supported is the Switch Independent. So you can’t implement LACP or other things with SET. Sorry network administrator’s guys J.

To finish you can configure the Load Balancing mode as standard NIC Teaming. But you have only two options: Hyper-V Port or Dynamic.

About Live Migration

In Windows Server 2016 Technical Preview 3, SET does not support Live-Migration (But it should be supported in the final release because official Microsoft Diagrams say the opposite).

SET, VM Queues and RSS

Because the only mode supported by SET is Switch Independent, the total number of VM queues in the team is the sum of the VM Queues of each NIC in the team. It is also called the Sum-Queues.

As standard NIC teaming in Windows Server 2012 R2, it is necessary to set the RssBaseProcNumber to at least 2 and MaxRssProcessor (Powershell command: Set-NetAdapterVmq). By setting the RssBaseProcNumber to at least 2, the logical Core 0 and 1 are not used for networking purposes and leave these cores for system processing.

Play with Switch Embedded Teaming

In this Technet Topic, Microsoft recommends to use Virtual Machine Manager to manage SET. However SET can be managed from PowerShell and you can find the commands on the above topic.

To test the SET management, I add in the team two NICs that I don’t use:

So I run the New-VMSwitch command by using EnableEmbeddedTeaming option:

Next I run Get-NetAdapterRdma to verify if I have Virtual NICs RDMA capable:

Then I enable RDMA on VMSwitchSET:

To finish, I remove the VMSwitch: