Category: Hyper-V

This one embarrassingly stopped me in my tracks for a day until I got some help from Willem Kasdrop in MS Netherlands on the Minasi Forum.  

The symptoms were as follows:

• I’d set up a VM in Hyper-V.
• I’d add it to the cluster.
• I’d finish the VM configuration, e.g. virtual network and VLAN ID.
• I’d test the quick migration or cluster fail over.
• I’d check the VM but the network configuration was reset to the defaults, the VLAN ID checkbox was cleared, the VLAN ID entry was back to "2" and the virtual switch was back to the original setting.

I went as far as rebuilding the cluster (that takes no time at all in W2008) but couldn’t find the solution.  Then Willem suggested that I hadn’t refreshed the cluster … as in right click the VM and select the entry to refresh the VM cluster configuration.

Why would you do this?  I was wondering when I checked the VM XML why I wasn’t seeing any virtual network configuration as above.  It turns out the cluster configuration for the VM stores some information.  If you change the VM configuration then you should:

• Update the cluster configuration.
• This saves the state of the VM, thus bringing it offline.
• You get a report on the update.
• You need to manually bring the VM back online.

After that, you’ll find that the configuration successfully follows the VM on fail over and fail back.

Anthony Crotty recently recorded a video with MS Ireland where he talks about Hyper-V.

I’ve just gotten my first Hyper-V cluster up and running.  It was really easy.  I’m doing a pilot build where the plan is to test like crazy, wipe the build and then put in a clean production system.  I used this plan for a multiple cluster ESX deployment this time last year and it worked out nicely.

My plan was to not activate the OS until as late as possible.  I found that none of the updates, e.g. Hyper-V RTM, Hyper-V cluster update, etc would install … I was being told "This update is not applicable for your system".  I’d built my Server Core host images by slipstreaming the update into install.wim for a WDS deployment to workaround that one.  But I really wanted that cluster fix so I could copy/paste the GUID for the VM LUN’s from the cluster MMC into the Hyper-V MMC to build the VM’s.  The lack of an activation caused that problem … easy to fix but annoying for anyone using OEM licensing with it’s limited activations.

Anyway, the clustering was easy and I’ve always thought Hyper-V was easy.  I’m impressed!

It looks like the Linux integration components for Hyper-V have gone RTM.  Integration services optimise the performance of guest operating systems on Hyper-V.  You can install and run x86 or x64 operating systems without them (especially if they are Xen enabled) but having these components will reduce their impact on the host and improve their performance.

The download is on Connect.  They appear RTM (kind like Network Monitor was RTM on Connect); they are version 1.0. 

Right now, the integration services support:

• SUSE Linux Enterprise Server 10 SP2 x86
• SUSE Linux Enterprise Server 10 SP2 x64

I’d expect to see RedHat coming fairly soon.  Microsoft is supporting RedHat with the OpsMgr Cross Platform Extensions and it wouldn’t make sense to have differing Linux support policies in the data centre.

However, you can still run RedHat on Hyper-V thanks to those Xen extensions.

We knew there was a specific build of Windows Server 2008 with Hyper-V (only) in the works.  It’s going to be an answer to the free VMware ESXi … a stripped down solution with no clustering functionality.  We expected there would be a cost to it … $28 plus whatever an OEM would charge.

Some details have been revealed.  A new SKU will RTM in less than 30 days and it will be free to download and use.  This is excellent for anyone wanting to get a cheap entry point into virtualisation on Microsoft’s platform.

Some things were released to the press by MS yesterday about the future of Hyper-V.  Things will be very, very exciting!

• Hyper-V will have "live migration" (what VMware calls VMotion).  This will be made possible by a new cluster file system that MS has developed.
• A new Core edition of Windows with just the Hyper-V role will be released pretty soon.  This will not support clustering.  It’ll be installed onto an OEM supplied (and probably customised) memory card that will be located internally in servers, much like is being done with ESXi.
• VMM 2008 will RTM within 1 month.  It was "launched" yesterday.  You will want to run VMM if running multiple Hyper-V hosts (at the least) or if clustering them.

I found a link to this on the MS Virtualisation Team Blog this morning.  It’s an article on things you should consider when backing up VM’s.  It’s well worth reading

There’s two ways you can approach backing up VM’s.  The first is to treat each VM as a normal server by installing an agent in the guest OS and using that to backup data, configurations and the system state.  This is the most comprehensive solution you’ll get. 

The other solution is to take a snapshot of the machine itself.  Hyper-V allows this by using Volume Shadow Copy Service (VSS).  A VSS writer stops the VM for a couple of milliseconds to capture it in a consistent state.  Consider a database being processed … do you want to snap the VM while things are happening?  Maybe you’ll get one table showing part of a transaction and another table not showing the other half of the transaction.  That’s not pretty at all!  This process used by VSS is also how you can get consistent VSS snapshots of SQL and Exchange.

Another thing to consider is virtualised DC’s.  Just like in VMware ESX, it’s recommended that you have at least one (preferably two) physical DC’s for your network to avoid potential chicken and egg scenarios.  But you might have other DC’s that are virtualised.  You should treat them just like physical DC’s.  Trying to do clever things likes snapshots and rollbacks of virtualised DC’s will lead to horrible scenarios like the USN rollback.

This article also mentions pass through disks not being supported by VSS.  If you think about it, of course they cannot.  There is no host file system when using a pass through disk to perform VSS.  The VM is using a raw LUN.  Be very, very careful about using SAN snapshots here; it’s not supported for a consistent state snapshot if the VM is running.

This is quite a frustrating problem.  The origin was that some of the XML files that define my VM’s were locked by AV on a test Hyper-V server.  This caused Hyper-V to fail to load the VM’s.  OK.  That’s not a biggie  – fix the problem and then start up the VM’s is what you’d think.  Uh-uh!  Hyper-V had managed to unload all knowledge of those VM’s.  OK – maybe I could just reopen the VMs?  Hyper-V does not have an "open" function.  Finally, I tried setting up a new VM with the existing config and disks.  That’s a monotonous manual task.  The problem here is that I lost my saved snapshots.

I opened a call with MS.  I also tried using the TechNet forums.  Let’s see what I got there before I get back to the call.  The MS staff on there gave me comments such as "why were you scanning?" and "don’t use snapshots in production".  Hmm.  Most people put AV on their servers.  The reason is to PROTECT them.  I’m sure if the Hyper-V team talked to the ForeFront team they’d be told the same thing.  I had put an exception in for the VHD files but didn’t know to do the same for the XML’s until it was too late.  However, even if I had, there’s always a chance a junior member of staff could accidentally override that exception.  What am I to do then, lose half of the VM’s on all of my hosts?  Is that acceptable to MS? 

I nearly choked when I saw a video of Steve Riley at TechEd talking about the potential of scanning VM’s using AV.  DONT TRY IT!

And as for using snapshots: A big use of virtualisation is setting up test environments.  Whether it’s for software development or infrastructure deployments, virtualisation earned it’s acceptance this way and continues to be a strong player there.  MS should understand this seeing as they think everyone in IT is a programmer.  Snapshots are a convenient way to get a VM back to a known state.  No tester will want to use backups for this because it’s too cumbersome, expensive and slow.

Back to the call.  After the usual chain of mails to get the engineer to understand the problem I finally got him on board.  In case anyone from MS is reading, here’s the case number: SRX080721602387.  I learned something from this call.  Hyper-V knows what VM’s and snapshots exist via shortcuts:

• %SYSTEMDRIVE%ProgramDataMicrosoftWindowsHyper-VVirtual Machines: This contains shortcuts to the XML files of each VM.
• "%SYSTEMDRIVE%ProgramDataMicrosoftWindowsHyper-VSnapshots: This contains shortcuts to the XML files of each snapshot.

Here’s where it gets nutty.  The missing VM’s were defined in these locations.  Now we had no idea why Hyper-V wasn’t loading my VM’s (reboots and all).  We tried recreating this shortcuts via Explorer and via command prompt.  No joy.

I eventually gave up because the ticket was now running around in circles.  I needed a working test lab back, despite losing my snapshots which will eventually lead me having to recreate my lab environment from scratch.

What to learn from this?  Hyper-V badly needs a simple way to open an existing VM.  Virtual Server has it.  Virtual PC has it.  VMware has it.  Why doesn’t Hyper-V?

I’ve been thinking about how I’m going to automate builds of Hyper-V servers in a Hyper-V cluster.  There’s lots of options now – I can create an image and sysprep it.  If I use diskless servers then I can snapshot that LUN in the SAN or I can use Ghost, ImageX or WDS to capture the image and deploy it.  Alternatively I can build up an unattended install using WAIK and scripts to slipstream the Hyper-V update and configure it.

We’re starting to see some posts hit the net on the subject now.  These 3 are interesting reads:

Sysprep and Hyper-V: Watch out because you still have some manual steps to repair networking and to enable the hypervisor after deploying your image.

Unattended Installation of Windows and Hyper-V: This might be the quick-hit solution but won’t be as fast as a sysprepped image when it comes to deployment.

Integrated Installation and The Beauty of the Win6 Servicing Stack: A continuation of the sysprep concept for WDS deployment.  I can see myself going down this route.

Why do I prefer Hyper-V, a version 1.0 hypervisor, over the more mature VMware ESX?  It’s quite simple; management.  This is where the "religious" VMware nutters scream about Virtual Center and ESXi web consoles – hold on to your hats, girls, this ones gonna be a bone shaker!

I am a "laxy admin".  I do not like to be poking and prodding in machines and consoles on a constant basis to do repetitive work.  There’s better things that I can be doing such as actual engineering projects or working on the business side of things.  I also like to know when something has gone wrong, either before it happens or before the customer calls us up.  The traditional solution seems to be to have lots of management consoles all over the place.  Honestly, that doesn’t work.  Once server and application crawl takes over, there’s too much fire fighting involved in working with lots of management solutions.

Here’s why I like HP (Dell play nice too AFAIK) and Hyper-V.  The fit in nicely with the concept of Optimised Infrastructure by being very manageable, more than their competition.  The idea is that you design your network, servers and applications so that they are easy to manage.  This means using integration and automation so there is less manual work to be done, the service is fault tolerant and reliable, you can focus on developing/enabling the business and the service that IT provides can be counted on.  We also reduce our operating costs.  Understanding these concepts and being able to use them is the difference between employing 15 IT staff and 77 IT staff (based on a real-world example).

So back on point … what’s all this got to do with Hyper-V?  MS’s System Center family of products are an integrated set of management tools to designed to build that automation and expertise into your network.  Yes, in the past they were MS centric but partners did expand them to include 3rd party solutions, e.g. *NIX and Cisco.  Now, MS is even doing this themselves.  One of the core products they sell is OpsMgr 2007, the monitoring solution.  Using an OpsMgr agent with management packs, I have expertise on different products that knows what to monitor, what is acceptable, what faults to watch for, best practices, etc.  I can even extend this or tweak it with exceptions.  This allows me to sit back and know that someone … or something … is watching my hardware, OS and applications. 

Here’s the fun bit.  There’s soon going to be a management pack for Hyper-V.  That means we get in-depth expertise for monitoring the health and performance of the virtualisation platform using the same single pane of glass that I use to monitor everything else.

So those VMware marketing types who try to sell ESXi off as being equal to Hyper-V, answer me this?  Where do I install an agent on a machine with no OS?  I’ve heard that I can monitor the hardware using cards in the server; what good is that for monitoring the hypervisor?  You answer me that the hypervisor has a web console.  Fantastic!  Do I really want to log into lots of little web consoles?  Ah … Virtual Center … so now I need to use it and my console that manages everything else?  Virtualisation is meant to be good for a lazy admin like me … you know .. less work, put my feet up, more time for playing games, etc. 

Microsoft’s answer to Virtual Center is Virtual Machine Manager 2008 which is being launched on September 8th.  VMM 2008 gives us management over the VM’s on our Hyper-V servers or cluster.  It includes the ability to audit physical machines to see if they’re candidates for virtualisation (don’t even have to pay for that agent license!) and a P2V conversion tool.  VMM 2008 integrates with OpsMgr 2007 SP1 via PRO or Performance and Resource Optimization.  You can read much more about that here.  The idea is simple.  OpsMgr monitors performance/health and understands the relationship between VM’s and hosts.  VMM 2008 manages VM creation and placement.  PRO links the two to share that knowledge and act on it.  What’s really cool is that we’re getting cradle-grave management of hosts and VM’s.  But not only at the hypervisor, but all the way through the "stack" from the hardware, the host virtualisation, the VM and the VM’s OS and applications.

That means I have a single integrated management solution for my entire network.  I’m a big believer in infrastructure optimisation.  I’ve witnessed it working and making my life easier.  I’ve also witnessed the opposite where there was no management despite there being lots of junkware being installed to "manage" points of the infrastructure.  Automation, expertise and integration are the keys to success.  For me, that’s why I like HP servers/storage and Hyper-V because they can be easily managed using Microsoft System Center.