I posted earlier today about my network transfer tests on HP ProLiant BL460C G5 blade servers with Windows Server 2008 R2 Hyper-V. Hans Vredevoort also did some tests, this time using BL460C G6 blades. This gave Hans the hardware to take advantage of some of the new technologies from Microsoft. Check out his results.
Tag: Windows Server 2008 R2
W2008 R2 Hyper-V Networking Enhancements
Windows Server 2008 R2 includes some enhancements to optimise how networking works in Hyper-V. I’m going to have a look at some of these now.
Virtual Machine Queue
Here’s the way things worked in Windows Server 2008. The NIC (bottom left) runs at the hardware level. VM1 has a virtual NIC.
When it communicates memory is copied to/from that NIC by the parent partition. All routing and filtering and data copying is done by the parent partition in Windows Server 2008.
Windows Server 2008 R2 takes advantage of Microsoft partnering with hardware manufacturers.
How it works now is that the NIC, i.e. the hardware, handles the workload on behalf of the parent partition. Hardware performs more efficiently than software. All that routing, filtering and data copy is handled by the network card in the physical host. This does rely on hardware that’s capable of doing this.
The results:
- Performance is better overall. The CPU of the host is less involved and more available. Data transfer is more efficient.
- Live Migration can work with full TCP offload.
- Anyone using 10GB/E will notice huge improvements.
Jumbo Frames
TCP is pretty chatty. Data is broken up and converted into packets that must be acknowledged by the recipient. There’s an over head to this with the data being encapsulated with flow, control and routing information. It would be more efficient if we could send fewer packets that contained more data, therefore with less encapsulation data being sent.
Jumbo Packets accomplish this. Microsoft claims that you can get packets that contain 6 times more information with this turned on. It will speed up large file transfers as well as reduce CPU utilisation.
Chimney Offload
This one has been around for a while with Windows but is support for Hyper-V was added with Windows Server 2008 R2.
It’s similar to VMQ, requiring hardware support, and does a similar job. The NIC is more involved in doing the work. Instead of offloading from the parent partition, it’s offloading from the Virtual Machine’s virtual NIC. The virtual NIC in the VM advertises connection offload capabilities. The virtual switch in the parent partition offloads child partition TCP connections to the NIC.
Hardware Reliance
You need support from the hardware for these features. During the RC release, the following NIC’s were included by MS in the media:
VM-Chimney Capable Drivers:
- Broadcom Net-Xtreme II 1 Gb/s NICs (Models 5706, 5708, and 5709)
- Broadcom 10Gb/s NICs (Models 57710, 57711)
VMQ Capable Drivers:
- Intel Kawela (E1Q) 1 Gb/s NICs (also known as Pro/1000 ET NICs)
- Intel Oplin NICs (IXE) 10Gb/s NICs (also known as 82598)
W2008 R2 Hyper-V Network Speed Comparisons
Hans Vredevoort asked what sort of network speed comparisons I was getting with Windows Server 2008 R2 Hyper-V. With W2008 R2 Hyper-V you get new features like Jumbo Frames and VMQ (Virtual Machine Queue) but these are reliant on hardware support. Hans is running HP G6 ProLiant servers so he has that support. Our current hardware are HP G5 ProLiant servers. I decided this was worth a test.
I set up a test on our production systems. It’s not a perfect test lab because there are VM’s doing their normal workload and thing like continuous backup agents running. This means other factors that are beyond my control have played their part in the test.
The hardware was a pair of HP BL460C “G5” blades in a C7000 enclosure with Ethernet Virtual Connects. The operating system was Windows Server 2008 R2. The 2 virtual machines were also running Windows Server 2008 R2. I set them up with just 512MB RAM and a single virtual CPU. Both VM’s had 1 virtual NIC, both in the same VLAN. They had dynamic VHD’s. The test task would be to copy the W2008 R2 ISO file from one machine to the other. The file is 2.79 GB (2,996,488 bytes) in size.
There were three tests. In each one I would copy the file 3 times to get an average time required.
Scenario 1: Virtual to Virtual on the Same Host
I copied the ISO from VM1 to VM2 while both VM’s were running on host one. After I ran this test I realised something. The first iteration took slightly longer than all other tests. The reason was simple enough – the dynamic VHD probably had to expand a bit. I took this into account and reran the test.
With this test the data stream would never reach the physical Ethernet. All data would stay within the physical host. Traffic would route via the NIC in VM1 to the virtual switch via its VMBus and then back to the NIC in VM2 via its VMBus.
The times (seconds) taken were 51, 55 and 50 with an average of 52 seconds.
Scenario 2: Virtual to Virtual on Different Hosts
I used live migration to move VM2 to a second physical host in the cluster. This means that data from VM1 would leave the virtual NIC in VM1, traverse VMBus and the Virtual Switch and physical NIC in host 1, the Ethernet (HP C7000 backplane/Virtual Connects) and then the physical NIC and virtual switch in physical host 2 to reach the virtual NIC of VM2 via its VMBus.
I repeated the tests. The times (seconds) taken were 52, 54 and 66 with an average of 57.333 seconds. We appear to have added 5.333 seconds to the operation by introducing physical hardware transitions.
Scenario 3: Virtual to Virtual During Live Migration
With this test we would start with the scenario in the first set of tests. We would introduce Live Migration to move VM2 from physical host 1 to physical host 2 during the copy. This is why I used on 512MB RAM in the VMs; I wanted to be sure the live migration end-to-end task would complete during the file copy. The resulting scenario would have VM2 on physical host 2, matching the second test scenario. I want to see what impact Live Migration would have on getting from scenario 1 to scenario 2.
The times (seconds) taken were 59, 59 and 61 with an average of 59.666 seconds. This is 7 seconds slower than scenario 1 and 2.333 seconds slower than scenario 2.
Note that Live Migration is routed via a different physical NIC than the virtual switch.
Scenario 4: Physical to Physical
This time I would copy the ISO file from one parent partition to another, i.e. from host 1 to host 2 via the parent partition NIC. This removes the virtual NIC, virtual switch and the VMBus from the equation.
The times (seconds) taken were 34, 28 and 27 with an average of 29.666 seconds. This makes the test scenario physical data transfer 22.334 seconds faster than the fastest of the virtual scenarios (scenario 1).
Comparison
| Scenario | Average Time Required (seconds) |
| Virtual to Virtual on Same Host |
52 |
| Virtual to Virtual on Different Hosts |
57.333 |
| Virtual to Virtual During Live Migration |
59.666 |
| Physical to Physical |
29.666 |
Waiver
As I mentioned, these tests were not done in lab conditions. The parent partition NIC’s had no traffic to deal with other than an OpsMgr agent. The Virtual Switch NIC’s had to deal with application, continuous backup, AV and OpsMgr agent traffic.
It should also be noted that this should not be a comment on the new features Windows Server 2008 R2 Hyper-V. Using HP G5 hardware I cannot avail of the new hardware offloading improvements such as VMQ and Jumbo Frames. I guess I have to wait until our next host purchase to see some of that in play!
This is just a test of how things compare on the hardware that I have in a production situation. I’m actually pretty happy with it and I’ll be happier when we can add some G6 hardware.
Lots Of Operations Manager Updates
Microsoft released lots of updates for Operations Manager over the last couple of weeks. There are lots of updates to management packs, too many for me to go posting them at this time of night. Have a look on the catalogue and you’ll see them. Or check your console if you’re using OpsMgr 2007 R2.
Most importantly is KB971541, Update Rollup for Operations Manager 2007 Service Pack 1.
“The Update Rollup for Operations Manager 2007 Service Pack 1 (SP1) combines previous hotfix releases for SP1 with additional fixes and support of SP1 roles on Windows 7 and Windows Server 2008 R2. This update also provides database role and SQL Server Reporting Services upgrade support from SQL Server 2005 to SQL Server 2008.
The Update Rollup includes updates for the following Operations Manager Roles:
- Root Management Server, Management Server, Gateway Server
- Operations Console
- Operations Management Web Console Server
- Agent
- Audit Collection Server (ACS Server)
- Reporting Server
The following tools and updates are provided within this update which may be specific to a scenario:
- Support Tools folder – Contains SRSUpgradeTool.exe and SRSUpgradeHelper.msi (Enables upgrade of a SQL Server 2005 Reporting Server used by Operations Manager Reporting to SQL Server 2008 Reporting Server)
- Gateway folder – Contains a MSI transform and script to update MOMGateway.MSI for successful installation on Windows Server 2008 R2
- ManagementPacks folder – Contains an updated Microsoft.SystemCenter.DataWarehouse.mp which requires manual import
For a list of fixes and tools addressed by this update rollup, see KB971541.
This update is supported for application on System Center Operations Manager 2007 Service Pack 1 only.
Feature Summary
The System Center Operations Manager 2007 SP1 Rollup 1 contains:
- All binary hotfixes released since Service Pack 1 release
- Support for Windows 7 and Windows Server 2008 R2
- Operational and DataWarehouse database support on Windows Server 2008 R2
- Additional stability hotfixes”
Requirements
- Supported Operating Systems: Windows 7; Windows Server 2003; Windows Server 2008; Windows Server 2008 R2; Windows Vista; Windows XP
- System Center Operations Manager 2007 Service Pack 1
Instructions
This update must be applied to each computer that meets the following criteria:
- Hosts a Microsoft Operations Manager Root Management Server
- Hosts a Microsoft Operations Manager Management Server
- Hosts a Microsoft Operations Manager Operations Console
- Hosts a Microsoft Operations Manager Web Console Server
- Hosts a Microsoft Operations Manager Reporting Server
- Hosts a Microsoft Operations Manager Manually installed Agent
- Hosts a Microsoft Operations Manager ACS Server
Before applying this update it is strongly recommended that Operations Manager databases, Management Server, Report Server and Web Console roles be backed up.
To extract the files contained in this update and installation of the update on the Operations Manager roles above:
- Copy the file – SystemCenterOperationsManager2007-SP1-KB971541-X86-X64-IA64-locale.MSI – To either a local folder or accessible network shared folder.
- Run the file – SystemCenterOperationsManager2007-SP1-KB971541-X86-X64-IA64-locale.MSI – locally on each applicable computer that meets the predefined criteria.
You can run SystemCenterOperationsManager2007-SP1-KB971541-X86-X64-IA64-locale.MSI from either Windows Explorer or from a command prompt. - Select the appropriate role to update from the Operations Manager 2007 Software Update dialog.
NOTE: To run this file on Windows Server 2008 you must run this file from a command prompt which was executed with the Run as Administrator option. Failure to execute this Windows installer file under an elevated command prompt will not allow display of the System Center Operations Manager 2007 Software Update dialog to allow installation of the hotfix”.
Upgrading Active Directory Domains to Windows Server 2008 and Windows Server 2008 R2 AD DS Domains
This guide explains the process for upgrading Active Directory domains to Windows Server 2008 and Windows Server 2008 R2, how to upgrade the operating system of domain controllers, and how to add domain controllers that run Windows Server 2008 or Windows Server 2008 R2 to an existing domain.
Upgrading your network operating system requires minimal network configuration and typically has a low impact on user operations. The upgrade process is straightforward, efficient, and allows your organization to take advantage of the improved security that is offered by the Windows Server 2008 and Windows Server 2008 R2 operating systems. This guide covers the process for upgrading domains and domain controllers, and how to add new domain controllers to existing Active Directory domains. It includes details about how to run Adprep.exe and resolve known issues and errors if they arise.
KB975354: W2008 R2 Hyper-V Rollup for CSV and VSS
Cluster Shared Volume (CSV) is the shared storage system that can be used by multiple cluster members at once for storing and running virtual machines in a Hyper-V cluster. CSV is specifically customised for Hyper-V and should not be used for anything else – you even have to agree to than to enable it. That customisation means that things would change a bit.
VSS is the Volume Shadow Copy Service. Using Hyper-V certified backup solutions like DPM you can backup the state of a virtual machine running in Hyper-V in a supported manner. This is done at the host level. That’s different to a file level backup that you would do with an agent installed in the VM. That would be able to recover individual files. The host level backup would be able to recover the entire VM back to the point of time that you did the backup.
There have been reports of issues. For example, DPM 2007 R2 is not live migration aware. You’ll have to wait until around April for DPM 2010 for a solution to that. That only affects the snapshot backups.
A rollup package has been released by Microsoft for Windows Server 2008 R2 Hyper-V to resolve some issues with CSV and VSS. Thanks to Hans Vredevoort (clustering MVP) for making me aware of this. Article KB975354 fixes the following situations.
“This update rollup package resolves some issues that occur when you backup or restore Hyper-V virtual machines
Issue 1
Consider the following scenario:
- Some Internet SCSI (iSCSI) connections are created in a virtual machine that is running Windows Server 2003.
- You back up this virtual machine on the virtual machine host server.
In this scenario, the error code 0x800423f4 occurs when you back up the virtual machine. Additionally, the following event is logged into the Hyper-V Virtual Machine Management Service event log:
The number of reverted volumes does not match the number of volumes in the snapshot set for virtual machine "’virtual machine name’ (Virtual machine ID <GUID>)".
Issue 2
Consider the following scenario:
- Cluster shared volumes are enabled on a failover cluster for Hyper-V.
- Some virtual machines are saved on the same volume. But they are running on different nodes.
- These virtual machines are backed up in parallel.
In this scenario, the virtual machine backup operation fails.
Issue 3
Consider the following scenario:
- A virtual machine is being backed up on a server that is running Hyper-V.
- At the same time, an application backup operation is being performed in the same virtual machine.
In this scenario, some data is truncated from the application backup in the virtual machine. Therefore, this behaviour causes data loss.
Issue 4
Consider the following scenario:
- A virtual machine that has some snapshots is backed up on a server that is running Hyper-V.
- Then, this virtual machine is restored to another location.
In this scenario, the restore operation fails and the virtual machine may be corrupted”.
If you’re running a Windows Server 2008 R2 Hyper-V cluster and are still getting used to it then there’s some good news. Here’s how I’ll approach this:
- Put the OpsMgr 2007 agent for host 1 into maintenance mode.
- Put the host 1 in maintenance mode in VMM 2008 R2. That kicks off live migration and moves VM’s from that host to another host. You can do this manually in the failover cluster management console if you don’t have VMM 2008 R2.
- Apply the update to host 1 and reboot.
- Test host 1 with a test VM.
- Repeat with all other hosts in the cluster.
That should work. And it’ll probably be your first opportunity to use Live Migration and VMM 2008 R2 Maintenance Mode in anger. Think about it, when do you normally get to do server patching during the work day? Now you can!