Month: November 2008

I’ll briefly mention some of the things I’m really liking that I was unaware of before putting VMM into action.

One of the things I was dreading with a VHD library was disk wastage.  PSS don’t like anything other than pass through disks and fixed size VHD’s in production.  Without VMM I was building sysprepped dynamically expanding VHD’s.  I’d store those in a shared folder.  I’d copy the image VHD to a host and then "convert" (it actually creates a new file) the VHD into a fixed size VHD which my new VM would use.

VMM is a little more clever.  It allows you to convert a disk in place.  I like that.  I’m storing my dynamic VHD’s in the library.  To save more space I’ve compressed the library – hey, disk is money to us and disk IS NOT CHEAP!  I can build a VM and convert the disks to fixed size before powering it up.  This means I can conserve disk space in the VMM library and still build fixed size VHD’s from templates without incurring nasty amounts of work.

Without Hyper-V using ISO images for the CD/DVD was a similar painful process -> copy the ISO to the Hyper-V box and load it up.  With VMM I can load the ISO into the library and it can be loaded on the VM via the VMM console over the network.

The idea of a template is different in VMM than it is in ESX … or even any OS deployment solution … more MS renaming!  It’ll take me a while to get used to but it mightn’t be a bad thing – I have to try it in anger first.  A template is a machine configuration, e.g. 1 processor, 2GB RAM, etc.  The VHD image is a totally different thing altogether.  So that 1 CPU & 2GB RAM machine description can be paired up with different OS images by the looks of it.

I’m seeing more and more how VMM makes managing multiple Hyper-V boxes easier.  It is different to ESX which I found quite natural (other than nested resource pools to be honest) but that difference isn’t naturally a bad thing.

BTW, after the early issues that I sorted out (and blogged about) it’s running very sweetly.  The diagram view went down very nice with the boss.  It’s nice to show the people in charge where all the money went 🙂

I alluded to a second problem with VMM 2008 and Hyper-V earlier.  The issue was that after a while, a previously healthy host would change to "requires attention".  The virtualisation status would change to unknown and the agent would stop communicating.  The host refresh would fail as follows:

Error (2927)
A Hardware Management error has occurred trying to contact server server.domain.local.
(Unknown error (0x80338104))

Recommended Action
Check that WinRM is installed and running on server server.domain.local. For more information use the command "winrm helpmsg hresult".

That WinRm error translates to "access denied".  I confirmed all the networking stuff and WinRM were actually OK.

Not long after that, every VM on that Hyper-V cluster would become unmanageable in VMM.  There’s a big clue that it’s a VMM issue.  Hyper-V and Failover Clustering stay healthy.  The VM’s are manageable in Hyper-V and run perfectly well.

Warning (13921)
Highly available virtual machine VM001 is not supported by VMM because one or more of its network adapters is not configured correctly. 

Recommended Action
Ensure that all of the virtual network adapters are either disconnected or connected to highly available virtual networks.

I’d set up PRO earlier that afternoon.  It was pretty simple.  An OpsMgr console is installed on the VMM server.  I installed PRO Tips on the OpsMgr 2007 SP1 server.  That also sets up the VMM console and the management packs on the RMS.  On the VMM server, I set up the FWDN of the OpsMgr server and the URL of the OpsMgr reporting server.  That’s it!

Everything was good when I left work.  When I got home I saw those above errors had taken place an VMM thought my cluster was messed up.  OpsMgr alerted me about the status of the VM’s.  Excellent!  I checked the supplied knowledge and it was more than I’d found in a day of googling.  As it turns out, my search terms sucked.  The provided expertise in the alert gave me the search term I needed and I found an excellent blog post on the issue.

I use Active Directory Group Policy restricted groups to control membership of the local administrators groups.  The VMM server was added by the agent install to the local administrators group to allow WMI and WinRM access.  My GPO would refresh after several hours and wipe out that group membership.  To fix this I reconfigured my GPO to add the VMM server to the Hyper-V host local administrators group and forced a GPO refresh on the server in question (GPUPDATE /FORCE).  I restarted WinRM (and VMM agent) on the affected host.  Finally I refreshed the host on the VMM server and the VM listings.  Everything was back to normal in just a few seconds.

Thank you PRO!

I’ve just configured the VMM 2008 web based portal.  It was pretty easy.  The idea is that you give people a web interface that allows them to manage VM’s, their properties and "KVM" access to them via the web site.  If you provide templates and VHD’s in your library (as well has prepared disk for your cluster) you can allow users to build their own VM’s.  To be honest, this would be impossible to control without a cluster file system – what’s to stop a user taking a 1TB LUN for a 100GB VHD?  You also are going to have trouble with restricting control over VLAN tags.  You can control VM resource consumption by using a points system, e.g. assign a score to a VM template and deduct it from a user’s point allocation as they deploy machines.

However, if you restrict full access to administrators and allow KVM/power control access to VM owners then you’ve got a nice solution.  You’ll want to do some clever group management and permissioning.

You’ll need an AD group for "VMM Administrators".  Put your VMM administrator accounts/groups into that group.  For every customer there will be a group, e.g. "Cust-Group".  They will also have a user called "Cust-User".  Cust-User and VMM-Administrators are members of Cust-Group. 

Create a folder/group in the VMM console to put that customer’s VM’s into called "Cust".  When a VM is set up for them assign the owner of the machine as "Cust-Group". 

Set up a self service role called "Cust-Self" service and add Cust-Group to that role.  Give it the scope required, i.e. the VMM folder/group called Cust.  Give the role the required permissions over the VM’s in that group, e.g. start, stop, pause & resume, remote connection and shutdown.

Now configure the portal with SSL access (simple IIS7 operation) and share the URL.  The user will log in using domainCust-User.  Their console will load an only show their VM’s.  They will only be able to do the actions you assigned to them.

My very early experiences were positive but the wrinkles are now only being worked out.  My advice for using VMM 2008 is this: deploy it before you deploy Hyper-V.  Adding an existing Hyper-V cluster to VMM 2008 is not recommended by me.

The first thing I’m going to talk about it virtual networks.  I have an existing Hyper-V cluster.  I created 2 virtual networks (mapped to 2 NIC’s) on each host.  Networking on very host was configured and named identically at every level by me.  The failover cluster validation report was a pass and VM’s failed over and back while maintaining network connectivity.  Perfect!

Then I added VMM 2008 to the mix.  It sucked in the cluster and deployed it’s agents.  The first problem (which I’m still trying to resolve) is a Win-RM access denied issue that appears after a host has been managed for several hours.  I’ve no idea why.  I’ve got a call open on this with MS so I hope to post a resolution at sometime soon.

The second problem is the one I’m going to talk about now.  After a while my highly available VM’s started going red with a status of "unsupported cluster configuration".  I knew the cluster was OK because of my report and because of how the VM’s moved OK.  I was getting an error on my VM’s telling me my networking was at fault.  Anthony Crotty sent me up a link that described a scenario when this error occurs.  It wasn’t identical but it did point out non-identical networking across the cluster hosts.  As far as I could see my networking was identical across all the hosts. 

There’s a property box for a Hyper-V cluster in VMM 2008.  In there you’ll find a networking tab to show networks that reside across all the hosts.  My 2 virtual networks weren’t there.  Intriguing!  I added a test private network to host 1 and it was automatically built across the other hosts.  That’s handy!

I deleted virtual network #2 and rebuilt it on host 1.  It was recreated on the other hosts and now it appeared in the cluster properties networking tab.

That leads me to this theory.  If you create virtual networks by hand (as you have to if you don’t yet have VMM 2008) then there’s a hidden configuration that’s required by VMM managed clusters that is not created.  I’m thinking there’s supposed to be a common ID in the hidden properties of the virtual network across the hosts.

So I recreated my 2 virtual networks, mapped the NIC’s and configured trunking on host 1 and they were created across the hosts.  I did a quick test, luckily.  VMM 2007 may set up the virtual networks on the other hosts but it did not map the NIC or configure trunking.  I replayed those steps on the other hosts.  You have to be patient when doing this.  You may see the dialog disappear when you click OK but there’s a job still running in the background to carry out the configuration.  Don’t jump straight back into the dialog box expecting to see your new configuration.  Watch the properties of the host update and wait for the job to complete.

Hopefully I can post something a little later about WinRM.  I expect it’ll be something stupid like the above.

One of the things that is critical to a virtualisation platform deployment is converting existing physical machines into virtual machines (P2V).  I attempted my first P2V last night and I really saw how much VMM 2008 adds to managing Hyper-V.  A lot of the work was done for me.

VMM 2008 P2V deploys a temporary agent to the physical machine (PM) to perform the conversion.  From what I’ve heard, this performs a VSS backup of the PM and send the data stream to be rebuilt on the VM.  It works a little like this:

• Deploy disk to your host/cluster.  In a cluster, note down the GUID of the disk being used.
• Find a nice quiet period where you can work on the PM.  I learned this the hard way when doing ESX P2V last year 🙂  People can get so mad when their server’s CPU and/or disk spike at 100% activity for hours on end 😉
• Start up the wizard.  You’ll need admin credentials for the PM and the name of the PM.
• Configure the VM: disk (you can set the type of VHD and make it bigger if required), CPU, RAM, power up/down and network.
• I leave the network disconnected and the machine set not to automatically power up.  This allows me to leave the PM running while testing the VM.  That’s not ideal in all scenarios, e.g. converting a mail or DB server.
• Pick a host.  If using a cluster, ensure that this is the host with the disk you’ve provided.
• In the disk configuration, ensure that the correct drive is selected.  I found it best to refresh the disk selection dialog if using GUID drives.   Otherwise the last second check would fail.
• Once the check passes, continue with the wizard.
• Now an agent on the PM copies the PM to the new VM.  The VM exists on the cluster at this point … the job will fail if the VMM service fails or the PM restarts so repeating the wizard will require deleting the VM.  The contents of the VM’s disk will get populated over the following hours.

I left that wizard running overnight.  I’ll be checking it when I get into work in a few hours.  My opinion so far is that it made managing GUID’s easier than bare Hyper-V.  It’s still behind Virtual Center because we don’t have that single storage made possible by a cluster file system such as VMFS.  However, that will come in Windows Server 2008 R2.