I say this a lot: “The great thing about Windows Server 2012 is that you have options”. It’s true, so very true.
Software can only ever do so much until it impacts the performance of the host. Hardware will always do things more efficiently, enabling lower latency, more throughput, and allowing greater scalability. Windows Server 2008 R2 recognised this by giving us some hardware offloads:
- Virtual Machine Queuing (VMQ) improved inbound networking for VMs
- Second Level Address Translation (SLAT – Intel EPT or AMD NPT/RVI) greatly improved the overall performance of memory paging intensive VMs by offloading VM to host memory mapping to the CPU rather than having the hypervisor do it.
This all continues in WS2012:
- Receive Side Scaling (RSS): Uses queues on the NIC and scales out beyond core 0 on the CPU to enable greater scalability for network processing
- Dynamic Virtual Machine Queuing (DVMQ): Using the same queues as RSS (and therefore being incompatible on the same NICs) it does something similar to RSS but for virtual machine traffic.
- IPsec Task Offload: Policy driven network encryption that is offloaded to capable NICs
- Datacenter Bridging (DCB): Protocol-based QoS can be offloaded to the NIC for non-virtual NIC traffic, giving better performance than by using the OS Packet Scheduler (and being able to do QoS and Priority Flow Control for RDMA)
Single Root I/O Virtualisation (SR-IOV) is a great example of the pros and cons of software versus hardware functionality. Without SR-IOV we have traditional virtual switch networking:
Packets come in the NIC, through the drivers, into the virtual switch for filtering and routing, back down the stack into the VM Bus (ring –1 and DEP/No Execute) and up into the virtual NIC in the VM. It’s software processing the entire way in and out again.
Now compare it with SR-IOV enabled on a host and VM:
You get something weird: the VM now has hardware connectivity to a Virtual Function (VF) on the NIC. There is on VF used for every vNIC and a pNIC has a number of VFs. Now software has been removed from the process of copying packets to the vNIC in the VM. Latency is improved, and the scalability of the host is improved too (more CPU available to apps in VMs).
The Pros of SR-IOV are easy to see. But do you need it? Maybe if you need the lowest possible latency. Maybe if you’re going to have hugely dense hosts or network processing could spike your CPUs. These are choices that we make for ESXi and Hyper-V. But where ESXi cannot vMotion one of these VMs, we can with Hyper-V. That’s nice.
On the con side, you cannot team the pNIC when usign SR-IOV because that NIC team (in the management OS) is completely bypassed. You’d have to create a NIC team in every VM … yuck! Double-yuck if deploying self-service clouds. So it is a trade-off.
You see a similar thing with DR replication. I love the concept of setting a SAN to replicate LUNs to another location and just leaving it. But it’s just not that simple.
On the pro side, SAN replication is set-and-forget (to some extent, assuming you monitor and manage by exception). That’s great for a cloud. VMs or services are put into one tier of storage and they auto-replicate to the DR site. Put into another tier and they are not. As a former hosting engineer, I love that.
On the con side, SAN replication is expensive (not so good in public cloud). It’s not only vendor lock in, but it’s usually SAN model lock in. I don’t love that. And while multi-suite clusters are a great idea, I think the WAN, physical networking, storage, servers, and virtualisation solution is one of the most complicated things you can build in IT infrastructure. While a consultant might be up to that, are the admins who are left holding the baby skilled enough to maintain it?
Hyper-V Replica is built-in and free to use. It’s very granular, and it’s designed to be able to work on commercial broadband. I love that. There’s no vendor lock in, and no requirement for partners. Another plus. It’s simple to set up and maintain. But there are scalability/performance considerations to any host-based replication. Hyper-V Replica also has resource requirements. That’s always the way with software. SAN based replication will not have those same host resource requirements, but it has other complexity/budgetary requirements.
Choosing the right solution for any site is a classic consultant’s “it depends”. There is never one right answer. Know the requirements of the customer/site, know the possible solutions, and find the best fit.
