How do you run multiple virtual machines on different subnets? Forget for for just a moment that these are virtual machines. How would you do it if they were physical machines? The network administrators would set up a Virtual Local Area Network or VLAN. A VLAN is a broadcast domain, i.e. it is a single subnet and broadcasts cannot be transmitted beyond its boundaries without some sort of forwarder to convert the broadcast into a unicast. Network administrators use VLAN’s for a bunch of reasons:
- Control broadcasts because they can become noisy.
- They need to be creative with IP address ranges.
- The want to separate network devices using firewalls.
That last one is why we have multiple VLAN’s at work. Each VLAN is firewalled from every other VLAN. We open up what ports we need to between VLAN’s and to/from the Internet.
Each VLAN has an ID. That is used by administrators for configuring firewall rules, switches and servers.
How do you tell a physical server that it is on a VLAN?
There’s two ways I can think of:
- The network administrators would assign the switch ports that will connect the server to a specific VLAN
- The network administrators can create a “trunk” on a switch port. That’s when all VLAN’s are available on that port. Then on the server you need to use the network card driver or management software to specify which VLAN to bind the NIC to. Some software (HP NCU) allows you to create multiple virtual network cards to bind the server to multiple VLAN’s using one physical NIC.
How about a virtual machine; how do you bind the virtual NIC of a virtual machine to a specific VLAN? It’s a similar process. I must warn anyone reading this that I’ve worked with a Cisco CCIE while working on Hyper-V and previously with another senior Cisco guy while working on VMware ESX and neither of them could really get their heads around this stuff. Is it too complicated for them? Hardly. I think the problem was that it was too simple! Seriously!
Let’s have a look at the simplest virtual networking scenario:
The host server has a single physical NIC to connect virtual machines. A virtual switch is created in Hyper-V to pass the physical network that is attached to that NIC to any VM that is bound to that virtual switch.
You can see above that the switch only operates with VLAN 101. Every server on the network operates on VLAN 101. The physical servers are on it, the parent partition of the host is on it, etc. The physical switch port is connected to the virtual machine NIC in the host using a physical network cable. In Hyper-V, the host administrator creates a virtual switch.
Network admins: Here’s where you pull what hair you have left out. This is not a switch like you think of a switch. There is no console, no MIB, no SNMP, no ports, no spanning tree loops, nada! It is a software connection and network pass through mechanism that exists only in the memory of the host. It interacts in no way with the physical network. You don’t need to architect around them.
The virtual switch is a linking mechanism. It connects the physical network card to the virtual network card in the virtual machine. It’s as simple as that. In this case both of the VM’s are connected to the single virtual switch (configured as an External type). That means they too are connected to VLAN 101.
How do we get multiple Hyper-V virtual machines to connect to multiple VLAN’s? There’s a few ways we can attack this problem.
Multiple Physical NIC’s
In this scenario the physical host server is configured with multiple NIC’s.
*Rant Alert* Right, there’s a certain small number of journalists/consultants who are saying “you should always try to have 1 NIC for every VM on the host”. Duh! Let’s get real. Most machines don’t use their GB connections in a well designed and configured network. That nightly tape backup over the network design is a dinosaur. Look at differential, block level continuous incremental backups instead, e.g. Microsoft System Center Data Protection Manager or Iron Mountain Live Vault. Next, who has money to throw at installing multiple quad NIC’s with physical switch ports all over the place. The idea here is to consolidate! Finally, if you are dealing with blade servers you only have so many mezzanine card slots and enclosure/chassis device slots. If a blade can have 144GB of RAM, giving maybe 40+ VM’s, that’s an awful lot of NIC’s you’re going to need :) Sure there are scenarios where a VM might need a dedicated NIC but there are extremely rare. *Rant Over*
In this situation the network administrator has set up two ports on the switches, one for each VLAN to connect to the Hyper-V host. VLAN 101 has a physical port on the switch that is cabled to NIC 1 on the host. VLAN 102 has a physical port on the switch that is cabled to NIC 2 on the host. The parent partition has it’s own NIC, not shown. Virtual Switch 1 is created and connected to NIC 1 and Virtual Switch 2 is created and connected to NIC 2. Every VM that needs to talk on VLAN 101 will be connected to Virtual Switch 1 by the host administrator. Every VM that needs to talk on VLAN 102 should be connected to Virtual Switch 2 by the host administrator.
Virtual Switch Binding
You can only bind one External type virtual switch to a NIC. So in the above example we could not have matched up two virtual switches to the first NIC and changed the physical switch port to be a network trunk. We can do something similar but different.
When we create an external virtual switch we can tell it to only communicate on a specific VLAN. You can see in the above screenshot that I’ve built a new virtual switch and instructed it to use the VLAN ID (or tag) of 102. That means that every VM virtual NIC that connects to this virtual switch will expect to be on VLAN 102 with no exceptions.
Taking our previous example, here’s how this would look:
The network administrator has done things slightly different this time. Instead of configuring the two physical switch ports to be bound to specific VLAN’s they’re simple configured trunks. That means many VLAN’s are available on that port. The device communicating on the trunk must specify what VLAN it is on to communicate successfully. Worried about security? As long as you trust the host administrator to get things right you are OK. Users of the virtual machines cannot change their VLAN affiliation.
You can see that virtual switch 1 is now bound to VLAN 101. Every VM that connects to virtual switch 1 will be only able to communicate on VLAN 101 via the trunk on NIC 1. It’s similar on NIC 2. It’s set up with a virtual switch on VLAN 102 and all bound VM’s can only communicate on that VLAN.
We’ve changed where the VLAN responsibility lies but we haven’t solved the hardware costs and consolidation issue.
VLAN ID on the VM
Here’s the solution you are most likely to employ. For the sake of simplicity let’s forget about NIC teaming for a moment.
Instead of setting the VLAN on the virtual switch we can do it in the properties of the VM. To be more precise we can do it in the properties of the virtual network adapter of the VM. You can see that I’ve done this above by configuring the network adapter to only communicate on VLAN (ID or tag) 102.
This is how it looks in our example:
Again, the network administrator has set up a trunk on the physical switch port. A single external virtual switch is configured and no VLAN ID is specified. The two VM’s are set up and connected to the virtual switch. It is here that the VLAN specification is done. VM 1 has it’s network adapter configured to talk on VLAN 101. VM 2 is configured to operate on VLAN 102. And it works, just like that!
Waiver: I’m seeing a problem where VMM created NIC’s do not bind to a VLAN. Instead I have to create the virtual network adapter in the Hyper-V console.
Here’s one to watch out for if you use the self servicing console. If you cannot trust delegated administrators/users to get VLAN ID configuration right or don’t trust them security-wise then do not allow them to alter VM configurations. If you do then they can alter the VLAN ID and put their VM into a VLAN that it might not belong to.
Firewall Rules
Unless network administrators allow it, virtual machines on VLAN 101 cannot see virtual machines on VLAN 102. A break out is theoretically impossible due to the architecture of Hyper-V leveraging the No eXecute Bit (AKA DEP or Data Execution Prevention).
Summary
You can see that you can set up a Hyper-V host to run VM’s on different VLAN’s. You’ve got different ways to do it. You can even see that you can use your VLAN’s to firewall VM’s from each other. Hopefully I’ve explained this in a way that you can understand.
Here is how to setup VLANs when using VMQ on Intel NICs.
http://www.intel.com/support/network/sb/CS-030993.htm
I find it much easier to just use the host OS and the NIC vendors utilities to create VLAN based network card instances. There is nothing to configure in Hyper-V manger other than adding another virtual switch and selecting the HOST based virtual NIC bound to the proper vlan. This removes all Vlan configuration from both Hyper-V and the Virtual Machines.
The only downside to this approach is if there are numerous vLans. Then I find it easier to create a virtual switch and bind it to a physical nic (or team) and do vLan tagging in the guest OS. The only downside to this is that anyone configuring the guest OS can potentially hop onto any vLan.
Where hardware guys get confused is that they are used to partitioning network segments at the switch and router level. It is harder for them to get their head around it being done at the Host OS, HyperVisor or Guest OS level.
There really is no right or wrong way to do it. I like doing it at the Host OS level as it removes all configuration steps at the lower levels, but then I only have four vLans to deal with and not 400. If I had 400 then I think I would be dealing with vLans at the guest OS level unless I needed to lock down security at a higher layer.
Most switches in large organizations today are using some form of automatic vLan registration, such as GVRP or similar to put the device on the right vLan. If this is the case then defining the vLans at the guest OS level is the easiest as it doesn’t require additional work from the host Virtual Machine administrator, but is dependent on the guest OS supporting vLan tagging on the virtual nic.
There is a wrong way to do it -> It’s your solution.
Agreed wit Adian.
One blog entry and it answered every question I could think of on this topic. Thanks for another excellent post Aidan, extremely helpful!
Great post. If I have only 1 VLAN for my production VM traffic and my network team configured the switch to use untagged since it has only 1 VLAN assigned to it does it matter at this point that I would not use VLAN number. Since the network team has isolated traffic at the switch level and I am not using teaming and I have 1 VLAN for all my VM traffic it seems fine that I would not put any VLAN on my hyper-v switch or in the guest OS. Is there any other reason I should change this?
If you’re using a flat network for VMs then there’s no need for tagging. That’s how it’s done for most SMEs.
how about the real interface shall i put the vlan id or not ?
See above.
I was looking a way to set up a lab where I have multiple sites in hyper-v.
How about installing a software router like GNS? That way, you can isolate completely from the network, but still VMs can be routed based on subnet.
I was thinking to buy a home VLAN switch or router to set up my lab, but still doing more research.
No idea to be honest. Never used it.
I’ve worked in a Cisco/VMware environment in the past and this is exactly the way we did it (I’m the network guy but was responsible for the network setup inside VMware as well). I’m new to a Hyper-V environment and this looks straight forward as well (not handling the Hyper-V side though). One question though: should management of the Hyper-V host itself be separated onto its own NIC, or can it be done on the same NIC(s) the VMs will be serviced from? Currently trying to deploy in a dev environment and NIC quantity is scarce, so we were attempting all parts from one NIC team but can’t seem to make it work.
It depends on your networking design. A modern design uses 10 GbE or faster NICs with converged networking, by implementing virtual NICs in the host’s management OS. These can be assigned to isolated VLANs, secure from the tenant workloads.
Hi. Nice article. A Question though. We use SCVMM2012R2 and I’d like to pass a trunk to my VM. I can do that with Powershell ofcourse:
Get-VMNetworkAdapter -vmname [VMNAME] | where {$_.MACAddress like “[MAC_ADDRESS]”} | Set-VMNetworkAdapterVlan -Trunk -NativeVlanId [NATIVE_VLAN_ID] -AllowedVlanIdList “[VLAN_LIST (Comma Separated)]”
But when I check in VMM the network configuration renders kind of invalid. After my PowerShell command I can no longer LiveMigrate the VM to another host because of the network configuration. Is there a supported way to configure this in VMM?
hi my friend, i’m trying to set up a mikrotik virtual machine on a hyper-v server, i did it the mikrotik is installed with 2 physical ports. which i need one to be connected to a dsl modem and the other one act as a trunk so i can create vlans in the virtual mikrotik to handle all the requests from the network.
its already configured but i have a problem with vlan tags since all the packets that come from the network and reaches to the trunk port of my switch should pass to the physical port of the server connected to the virtual nic of the mikrotik and should keep the tags so mikrotik can handle the tagging/untagging and send the traffic.
But the problem is i can see all the packets leaving from one of the vlan interfaces leave with the native vlan tag (1) and don’t keep the tags of the correct vlan.
do u know how could i solve this ??
Simplify. Do not connect directly to the router. Instead, route your VLANs (as required) to the router. That way only trunk ports are presented to the virtual switch.
If I wanted to created a virtualised firewall where the LAN vNIC would be handling routing for a number of VLANs, when I create the VM and assign the LAN vNIC to the Hyper-V “trunk” NIC, would I leave the VLAN ID in the VM’s vNIC settings blank and create the VLAN intrfaces at the OS level in the virtualised firewall?
You need to point me at a diagram.
Aidan can you tell me why Matt’s way is wrong?
Using OEM provided NIC teaming is not and never has been supported by Microsoft. Use what is in the OS – it works, it’s stable, and it is -supported-. If you want to walk a road alone, then you are a-l-o-n-e. And good luck down in hell, don’t bother calling me.
I have two physical switches stacked for redundancy. My hyper V host has two physical connections, one to each switch, which are configured for LACP and as a trunk with three VLANs. I want to expose two of these VLANs to each of my VM guests. It appears my best/only option is to configure LACP Nic teaming (OS native) on the host and to create VLAN interfaces (OS native) on the host for each VLAN. Then in hyper V I must create two virtual switches to the specific host’s VLAN interfaces. The guests will be configured with two nics, one to each virtual switch. Is this correct and best?
See above: trunk ports, tag the VM NICs. IT IS SIMPLE.
Thank you for your reply!
But seems the “simple” solution precludes the host from participating in all but the DEFAULT VLAN on the trunk (which may be a problem for me), correct?
If the host is converged, then assign a VLAN tag to the host’s vNIC. People have been doing this stuff since W2008 – if you’re still having trouble then you should bring an expert on site.
Hey excellent clarification on where the vlan tagging can be set. Can you comment a little on performance when configuring vlan on the physical switch port vs the host switch vs the VM adapter.?
Configure it on the settings of the vNIC.
Very useful article, Thank you.
Using Hyper-v switch setting is much easier than using the NIC’s utility (from my point of view).
Regards
Correct me if I’m wrong but the following below might be wrong;
‘When we create an external virtual switch we can tell it to only communicate on a specific VLAN. You can see in the above screenshot that I’ve built a new virtual switch and instructed it to use the VLAN ID (or tag) of 102. That means that every VM virtual NIC that connects to this virtual switch will expect to be on VLAN 102 with no exceptions.’
The VMs won’t be all on VLAN 102, it’s just you’re specifying the host that it will be on VLAN 102. At the end of the day, it’s still is a trunk mode which is carrying all the VLANs and the VLAN ID on the VMs still apply. From my understanding, specifying ID 102 means that a virtual port has been created on the virtual switch for the host and it’s on VLAN 102
The VMs will only be on VLAN 102 provided that on the switch you configured an access mode rather than a trunk.
Like I said, correct me if I’m wrong.
Hi Icaris.
You are not wrong, that screenshot and associated text is just plain wrong.
The vlan 102 set there is for the vNIC in the parent OS created when you make the vswitch. It has absolutely no effect on VMs connected to the vswitch.
Hi,
I am having some trouble here but I think I am stuck at the switch and sonic wall setup. I want to set up this scenario but my VM cannot see the Internet. Do I need to setup the port that the server is connected to as a Trunk and tag it with all the IDs that is coming out of the server? Then do the same with the port that is connected to the sonic wall?
is this possible without a managed switch and 1 physical nic?
Thanks!