Hyper-V converged networking and storage design

Since Windows Server 2012, the converged networking is supported by Microsoft. This concept enables to share an Ethernet adapter for several network traffics. Before that, it was recommended to dedicate a network adapter per network traffic (backup, cluster and so on).

So thanks to the converged networking, we can use a single Ethernet adapter (or teaming) to carry several network traffics. However, if the design is not good, the link can quickly reach the bandwidth limit. So when designing converged networking, keep in mind the QoS (Quality of Service) setting. This is this setting which will ensure that the traffic will have the appropriate bandwidth.

When you implement the converged networking, you can play with a setting called QoS weight. You can assign a value from 1 to 100. More the value is high; more the traffic associated with this value has priority.

When you design networks for Hyper-V/VMM, you have usually four networks for hosts: Host fabric Management, Live Migration, Cluster and Backup. I have detailed some examples in the next part Common Network requirements. The other network traffics are related to Virtual Machines. Usually you have at least a network for the fabric Virtual Machines.

Common network requirements

Host Management networks

In the below table, you can find an example of networks for the Hyper-V Hosts. I have specified the VLAN and the QOS Weight also. The Host Fabric Management has a VLAN number set to 0 because packets will be untagged. In this way, even if my Hyper-V host has no VLAN configuration, it can answer to DHCP request. It is useful to deploy host by using Bare-Metal from Virtual Machine Manager.

In the above configuration, Live-Migration and Backup traffics have a better priority than Host Fabric Management and Cluster traffics. It is because Live-Migration and Backup require a larger bandwidth.

VM Workloads

In the below table, you can find example of VM networks. In this example, I have isolated the network for the Fabric VMs, DMZ VMs and their cluster en backup traffics. In this way I can apply a QoS setting for each type of traffic. Here, Backup traffics have a higher weight than other networks because backup traffics use a larger bandwidth.

Hyper-V converged networking and storage designs

Now that you have your network requirements on paper, we can work on the storage part. First you have to choose the storage solution: FC SAN, iSCSI SAN or Software-Defined Storage?

To choose the storage solution you must look at your needs and your history. If you have already a FC SAN with good performance, keep this solution to save money. If you start a new infrastructure and you want to store only VMs on the storage solution, maybe you can implement a Software-Defined Storage.

In the next sections, I have drawn a schema for each storage solution usually implemented. They certainly did not suit all needs but they allow understanding the principle.

Using Fibre Channel storage

Fibre Channel (not fiber-optic cables) is a protocol used to connect a server to the storage solution (SAN: Storage Area Network) with high-speed network. Usually fiber-optic cables are used to interconnect the SAN with the server. The adapters where are connected the fiber-optic on the server are called HBA (Host Bus Adapter).

In the below schema, the Parent Partition traffics are represented by green links while VMs traffics are orange.

On Ethernet side, I implement two dynamic teaming with two physical NICs each:

• Host Management traffics (Live-Migration, Cluster, Host Backup, host management);
• VM Workloads (VM Fabric, VM DMZ, VM Backup and so on).

On the storage side, I split also Parent Partition traffics and VM traffics:

• The Parent Partition traffics are mainly related to Cluster Shared Volume to store Virtual Machines;
• The VM traffics can be LUN mounted on VMs for Guest Cluster usage (Witness disk), database servers and so on.

To mount LUN directly on VMs, you need HBA with NPIV enabled and you need also to create vSAN on Hyper-V host. Then you have to deploy MPIO inside the VMs. For more information, you can read this TechNet topic.

To support the multi-channel on the parent partition, it is also necessary to enable MPIO on the Hyper-V host.

For a production environment, you need four 10GB Ethernet NICs and four HBA. This is the most expensive solution.

Using iSCSI storage

iSCSI (Internet Small Computer System Interface) is a protocol that carries SCSI commands over IP networks from the server to the SAN. This solution is less effective that Fibre Channel but it is also less expensive.

The network design is the same that the previous solution. Regarding the storage solution, I isolate the parent partition traffics and the VM workloads. MPIO is implemented for CSV to support Multi-Channel. When VMs need direct access to storage, I deploy two NICs bound on each VM Volumes physical NICs. Then I deploy MPIO inside the VMs. To finish, I prefer to use dedicated switches between hosts and SAN.

For each Hyper-V hosts, you need eight 10GB Ethernet Adapter.

Using Software-Defined Storage

This solution is based on software storage solution (as Scale-Out File Servers).

The network is the same as previous solutions. On the storage side, at least two RDMA NICs capable are required for better performance. SMB3 over RDMA (Remote Direct Memory Access) enables to increase throughput and to decrease the CPU load. This solution is also called SMB Direct. To support Multipath, the SMB Multichannel must be enabled (not teaming!!).

When VM needs a Witness disk or other shared volume for Guest Clustering, it is possible to use Shared VHDX to share a virtual hard drive between virtual machines.

This solution is less expensive because the software-defined storage is cheaper than SAN.

What about Windows Server 2016

In Windows Server 2016, you will be able to converged NIC across tenant and RDMA traffic to optimize costs, enabling high performance and network fault tolerance with only 2 NICs instead of 4.