Microsoft brings a new feature to Hyper-V in Windows Server 2016 called VHD Set. This type of disk enables to share virtual hard disks between several servers to implement guest cluster. In this topic we will see why using VHD Set, and how to implement it.
Why using VHD Set instead of shared VHDX
As VHD Set, Shared VHDX enables to share a virtual hard disk between multiple virtual machines. This feature is useful to implement a guest cluster where shared disks are required (as SQL Server AlwaysOn FCI or File Servers). Shared VHDX and VHD Set are great to avoid the use of virtual HBA and virtual SAN to present a LUN to the VMs. They are also necessary if you have implemented a SMB3 based storage solution. However shared VHDX feature as some limitation:
- Resizing and migrating a shared VHDX is not supported
- Make a backup or a replica of a shared VHDX is not supported
The VHD Set feature has not these limitations. However, VHD Set is available only for Windows Server 2016 guest operating system. When creating a VHD Set, two files are created:
- A .avhdx file that contains data. This file is fixed or dynamic;
- A .vhds file that contains metadata to coordinate information between guest cluster nodes. The size of this file is almost 260KB.
Create a VHD Set
To create a VHD Set, you can use the Graphical User Interface (GUI) or PowerShell cmdlets. From the GUI, open the Hyper-V Manager, select New and then Virtual Disk. As below screenshot, select VHD Set.
Then select the type of disk (fixed or dynamic), the name, the location and the size. By using PowerShell, you can run the below cmdlet:
Below you can find the result of these last two actions:
As you can see, the “blue” VHD Set is fixed and the AVHDX file size is 40GB. The “red” one is dynamic and so AVHDX file will expend its size dynamically.
Add VHD Set to virtual machines
To try VHD Set, I have created two virtual machines called VMFLS01 & VMFLS02. Each VM will be connected to two VHD Sets:
- Quorum: for the cluster Witness disk
- Shared: for the data
To mount the shared disk into the VM, edit the VM properties and navigate to a SCSI controller. Then select Shared drive.
Next, specify the location of the VHDS file.
You can also mount the VHDS in VM by using PowerShell:
Add-VMHardDiskDrive -VMName VMFLS01 -Path " c:\ClusterStorage\VMStorage01\SharedDisk\VMFLS_Quorum.vhds" -SupportPersistentReservations Add-VMHardDiskDrive -VMName VMFLS01 -Path " c:\ClusterStorage\VMStorage01\SharedDisk\VMFLS_Shared.vhds" -SupportPersistentReservations
I have repeated the same steps for the second VM. Once both VMs are connected to the VHD Set, you can start the VM.
Add-VMHardDiskDrive -VMName VMFLS02 -Path " c:\ClusterStorage\VMStorage01\SharedDisk\VMFLS_Quorum.vhds" -SupportPersistentReservations Add-VMHardDiskDrive -VMName VMFLS02 -Path " c:\ClusterStorage\VMStorage01\SharedDisk\VMFLS_Shared.vhds" -SupportPersistentReservations
Create the guest cluster
Now that both VMs are connected to the shared disk, we can create the cluster. I run the following cmdlets to install required features on each server:
# Install Failover Clustering feature and management tools install-windowsfeature -Name Failover-Clustering -IncludeManagementTools -ComputerName VMFLS01 install-windowsfeature -Name Failover-Clustering -IncludeManagementTools -ComputerName VMFLS02
Then I execute the following command to make online disks and initialize them:
get-disk |? OperationalStatus -Like "Offline" | Initialize-Disk
Now that disks are initialized, I create a partition on each disk. In the above example, the disk 1 is for Quorum usage and the disk 2 for data usage.
New-Volume -DiskNumber 1 -FileSystem NTFS -FriendlyName Quorum New-Volume -DiskNumber 2 -FileSystem NTFS -FriendlyName Data
Next I run the following cmdlets to create the cluster:
# Test the nodes to check if they are compliant to be part of a cluster Test-Cluster VMFLS01,VMFLS02 # Create the cluster New-Cluster -Name Cluster-FS01 -Node VMFLS01,VMFLS02 -StaticAddress 10.10.0.199 # I rename Cluster Disk 1 to Quorum and Cluster Disk 2 to Data (Get-ClusterResource |? Name -like "Cluster Disk 1").Name="Quorum" (Get-ClusterResource |? Name -like "Cluster Disk 2").Name="Data" # Set the Cluster Quorum to use disk Witness Set-ClusterQuorum -DiskWitness Quorum # Set the Data volume to Cluster Shared Volume Get-ClusterResource -Name Data | Add-ClusterSharedVolume
Once you have finished, you should have something like this in the cluster:
Now I’m able to copy data in the volume:
I can also move the storage to another owner node:
Conclusion
Thanks to VHD Set in Windows Server 2016, I can create easily Guest Cluster without using complex technologies as NPIV, virtual HBA and virtual SAN. Moreover, the resizing, the migrating and the backup is supported when implementing shared disks with VHD Set. It is a friendly feature, so why not using it?
VHD Set is available only for Windows Server 2016 guest operating system.
is this still true
Unfortunately, yes 🙂
Is there any way to work with VHD Set volumes from VMM 2016?
I’m not sure that VMM 2016 enable to create a VHD Set. I think they work with Shared VHDX.
Haven’t found any way to work with shared VHDX through VMM 2016. If you figure it out, it would make a very interesting post.
Also…is there any way to migrate storage on VMs with VHD Sets?
Hi,
I don’t understand your question. You want to migrate a shared VHDX to VHD Set ?
Let me re-ask 🙂 Is it possible to do Storage live Migration (From CSV to other CSV) with VM’s that have VHDS?
No
Storage live migration for VHDSet still not supported in Windows 2016. It doesn’t even support migration when VM is offline (powered off).
I cant connect vhds volume to any server
I am testing 2 windows server 2016 virtualmachines on free Hyperv 2016 which i set on old Power:Edge server – created vhds on local drive and when i try to connect it to VM i get error
Add-VMHardDiskDrive : Failed to add device ‘Virtual Hard Disk’.
The storage where the virtual hard disk is located does not support virtual hard disk sharing.
When i try to move it to smb share (to my Win10 Workstation) it keep saying access denied (i set full fight to my account, hyperv machine account and 2virtual machines accounts
Hi, run “fltmc.exe attach svhdxflt x:\” in cmd if you are curious about testing it in a lab environment. It will let you attach the vhd sets to multiple VMs.
Hi,
The VHD Set must be located on shared LUN/Volume such as a CLuster Shared Volume. This is a limitation of VHD Set for the moment.
Hi Romain. what is the advantage if you have a shared disk in an SQL alwayson Scenario? I made a cluster with 3 nodes and a FS Witness without shared disk.
Because you implement AlwaysOn AAG. If you deploy AlwaysOn FCI, you need shared storage 🙂
HI. If we use the shared disk for SQL Dumps (SQL not likes UNC) for multiple hosts, what happened if the shared Disk is not available for the VM? would the VM not start because missing disk – just like all other disks?
If your SQL needs the disk (shared storage or standard storage), your service will not start.
i m planning to make sql cluster on 2016 , what do you think regarding of the performnace of vhd set ? is it better than shared vhdx , cause shared vdx in 2012 was very slow comparing if u attach the lun directly
I never tried VHD Set in production but you’ll not loose performance between shared VHDX and VHD Set. I recommend you heavily VHD Set. Try it and test performance before deploying in production.
Regards,
I’ve been trying to set up a lab regarding this technology and I dont understand how you managed to have the set on a LUN/Shared volume if you were storing them on the “C:\” drive. I’d like to learn more about this technology but I think you need to do a better job of listing what the requirements are for utilizing this technology. For instance how does the Hyper-V configuration validate that the storage in on a CSV? Is active directory required? I know 2016 introduced some possibilities for clustering without AD could this be utilized here?
Can you show the replication and backup via SCDPM 2016 process with VHD Set? im trying to but getting the error that says replication to Shared volumes not allowed
Hi,
Really I don’t know. I use Veeam instead DPM for 2 years 🙂
Can I know how to backup SQL CLuster using Veeam with VHDs?
Do the “Disk” related commands need to be run on both servers or just one to set them up and the other sees it once they are clustered?
The disk commands must be run on a single server because it’s a clustered disk. You have just to set in online in the second node.
Is there really no way to move VHD set from its CSV to a shared folder for example? I need to reinstall a host cluster which is hosting a “guest cluster”, using VHD set, but would like to save this guest cluster. If there is no VHDS “storage” migration supported, can I export/backup/replicate this guest cluster to somewhere else, just the time for me to reinstall the physical cluster and then restore it?
Which kind of backup is supported for VHD Set?
Do I have to backup it as a normal file at host level or can I consider it like a normal VM disk’s backup?
I’ve tried to use Veeam and DPM but I always have the same problem that cause me a disk corruption andd inability to snapshot the disk, but I have only tried to backup is as a VM, not as single files from host.
Is this the way?