Windows 10 Build 10565 Makes Nested Hyper-V Virtualisation … Possible!

One of the biggest hitting articles on my site, written in 2009 (!!!) is “Can You Install Hyper-V in a VM?”. The short answer has always been “yes, if you know how”, but the long/compelete answer continues with “the hypervisor will not start and you will not be able to boot any virtual machines”.

This was because Hyper-V did not support nested virtualization – the ability to run Hyper-V in a VM that is running on Hyper-V (yes, I know there are hacks to get Hyper-V to run in a VM on VMware). A requirement of Hyper-V is a processor feature, VT-x from Intel or AMD-V from AMD. Hyper-V takes control of this feature and does not reveal it to the guests running on the host. This means that a system requirement of Hyper-V is not present in the virtual machine, and you cannot use the virtual machine as a real host.

Microsoft released Build 10565 of Windows 10 to Windows Insiders this week and announced that the much anticipated nested Hyper-V virtualization is included. Yup, I’ve tried it and it works. Microsoft has made this work by revealing processor virtualization on a per-VM basis to VMs that will be Hyper-V hosts – let’s call these VM hosts to keep it consistent with the language of Windows Server Containers. This means that I can:

1. Install Hyper-V on a physical host
2. Create a VM
3. Enable nested virtualization for that VM, making it a VM host
4. Install a guest OS in that VM host and enable Hyper-V
5. Create VMs that will actually run in the VM host.

Applications of Nested Virtualization

I know lots of you have struggled with learning Hyper-V due to lack of equipment. You might have a PC with some RAM/CPU/fast disk and can’t afford more, so how can you learn about Live Migration, SOFS, clustering, etc. With nested virtualization, you can run lots of VMs on that single physical machine, and some of those VMs can be VM hosts, in turn hosting more VMs that you can run, back up, migrate, failover, and so on (eventually, because there are limitations at this point).

Consultants and folks like me have struggled with doing demonstrations on the road. At TechEd Europe and Ignite, I used a VPN connection back to a lab in Dublin where a bunch of physical machines resided. I know one guy that travels with a Pelicase full of of Intel NUC PCs (a “cloud in a case”). Now, one high spec laptop with lots of SSD could do the same job, without relying on dodgy internet connections at event venues!

A big part of my job is delivering training. In the recent past, we nearly bought 20 rack servers (less space consumed than PCs, and more NICs than NUC can do) to build a hands-on training lab. With a future release of WS2016, all I need is some CPU and RAM, and maybe I’ll build a near-full experience hands-on training lab that I can teach Hyper-V, Failover Clustering, and SOFS with, instead of using the limited experience solution that Microsoft uses with Azure VMs (no nested virtualization at this time). Personally I think this feature could revolutionize how Hyper-V training is delivered, finally giving Microsoft something that is extremely badly required (official Hyper-V training is insufficient at this time).

Real world production uses include:

• The possibility of hosted private cloud: Imagine running Hyper-V on Azure, so you can do private cloud in a public cloud! I think that might be pricey, but who knows!
• Hyper-V Containers: Expected with TPv4 of WS2016, Hyper-V Containers will secure the boundaries between containerized apps.

It’s the latter that has motivated Microsoft to finally listen to our cries for this feature.

Release Notes

• Nested virtualization is a preview feature and not to be used in production.
• AMD-v is not supported at this time. Intel VT-x must be present and enabled in the physical host.
• You cannot virtualize third-party hypervisors at this time – expect VMware to work on this.
• The physical host and the VM host must be running Build 10565 or later. You cannot use Windows 10 GA, WS2012 R2 or WS2016 TPv3 as the physical host or the VM host.
• Dynamic Memory is not supported.
• The following features don’t work yet: Hot-memory resize, Live Migration, applying checkpoints, save/restore.
• MAC spoofing must be enabled on the VNIC of the VM host.
• Virtual Secure Mode (VSM) / Virtualization Based Security (VBS) / Credential Guard (a Windows 10 Enterprise feature) must be disabled to allow virtualization extensions.

Enabling Nested Virtualization

1 – Install the Physical Host

Install Build 10565 of Windows or later on the physical host. Enable the Hyper-V role and configure a virtual switch.

2- Create a VM Host

Deploy a VM (static RAM) with Build 10565 or later as the guest OS. Connect the VM to the virtual switch of the physical host.

3 – Enable Nested Virtualization

Run the following, using an elevated PowerShell window, on the physical host to execute the enablement script (shared on GitHub):

Invoke-WebRequest https://raw.githubusercontent.com/Microsoft/Virtualization-Documentation/master/hyperv-tools/Nested/Enable-NestedVm.ps1 -OutFile ~/Enable-NestedVm.ps1

~/Enable-NestedVm.ps1 -VmName <VmName>

4 – Enable MAC Spoofing

Run the following on the physical host, targeting the VM host. This will enable MAC spoofing on the VM host. Modify this cmdlet to specify a vNIC if the VM will have NIC just for nested VMs to communicate on.

Set-VMNetworkAdapter -VMName <VMName> -MacAddressSpoofing on

5 – Enable Hyper-V in the VM Host

Enable the Hyper-V role in the VM host and configure a virtual switch on the vNIC that is enabled for MAC spoofing.

6 – Create Nested VMs

Create VMs in the VM host, power them up and deploy operating systems. Have fun!

And bingo, there you go!

How Useful is Nested Virtualization Now?

I won’t be rushing out to buy a new laptop or re-deploy the lab yet. I want to run this with WS2016 so I have to wait. I’ll wait longer for Live Migration support. So right now, it’s cool, but with WS2016 TPv4 (hopefully), I’ll have something substantial.