Month: April 2013

Understanding Hyper-V Error Messages

Ever been confronted by a Hyper-V error box like this?

image

This one isn’t a great example because the error is pretty clear.  But the above error box illustrates the sort of information that is returned from the management layers of the Hyper-V management OS (which runs in the parent partition).  What does it all mean?  Where does it come from?

Understanding Hyper-V Management

Take a wander over to MSDN to re-familiarise yourself with Hyper-V’s architecture.  It’s always good to revisit and refresh yourself with how Hyper-V works because this information can make troubleshooting much easier.

See what the page calls the root partition?  That’s actually the parent partition, where you’ve installed the host’s OS, which then became the management OS when:

  1. You enabled Hyper-V
  2. The physical machine rebooted
  3. The type 1 hypervisor slipped itself in between Windows Server and the hardware

Let’s focus on 3 pieces that run in User Mode in the management OS:

  • WMI: Everything that you do to manage Hyper-V should go through the Hyper-V WMI.  All the MSFT tools do this: Hyper-V Manager, Failover Cluster Manager, PowerShell, and VMM.  3rd party tools should do the same.  Doing otherwise can … cause issues.  WMI is the interface to managing Hyper-V and the VMs on the host.
  • VMMS: The Virtual Machine Management Service … manages Hyper-V.
  • VMWP: There is one Virtual Machine Worker Process for each running virtual machine.  It’s a small process called VMWP.EXE.  It’s involved with all sorts of things for managing the VM: taking part in Live Migration, managing the state transitions of the VM (start, stop, shutdown, etc), Dynamic Memory, and so on.

Event Logs

Each module produces its own information and logs it in the Windows event logs.  Open up event viewer and browse to Applications And Service Logs > Microsoft > Windows and you can see a series of folders for Hyper-V components:

  • Hyper-V-Config
  • Hyper-V-High-Availability
  • Hyper-V-Hypervisor
  • Hyper-V-Integration
  • Hyper-V-SynthFC
  • Hyper-V-SynthNic
  • Hyper-V-SynthStor
  • Hyper-V-VID
  • Hyper-V-VMMS
  • Hyper-V-Worker

Ben Armstrong (@VirtualPCGuy) blogged about these back in 2009.  You’ll notice some changes in WS2012 since then, such as the addition of a log for virtual Fibre Channel events (Hyper-V-SynthFC).

As Ben notes, Hyper-V-VMMS is the place to start looking when there’s a problem with Hyper-V.  It gives us the WMI interface into Hyper-V.  And Hyper-V-Worker is a good place to look if you’re troubleshooting anything to do with the responsibilities of a Worker Process.

Understanding The Error

What you see in the error box is an accumulation of information from several sources.  In fact, you’ll find some of this text in the event logs.  To demonstrate this, I have:

  • Created a virtual machine with a VHDX file
  • Deleted the VHDX file
  • Attempted to start the virtual machine

Below, I have expanded the resulting error to see much more information:

image

Let’s start with the text blocks that are highlighted in red, working our way from the bottom one to the top.  These are errors related to the VMWP-managed state transition of the VM.  As a result, these text blocks are errors that are logged by the Worker Process in Hyper-V-Worker.

The bottom error, which was the first to be logged by the VMWP, is here:

image

Note that the text is identical.  This continues with the subsequent VMWP text blocks & error log entries:

image

And again:

image

Now onto the text block that I’ve highlighted in green.  That you can find in the Hyper-V-VMMS log:

image

Read from top to bottom, the story is:

  • The virtual machine could not start
  • The IDE controller had a problem
  • The VHDX file could not be found
  • The folder that the VHDX is supposed to be in is OK, but Hyper-V could not find the specified file

The Lessons To Learn

There are 3 things to pick up from this post:

  1. Get familiar with Hyper-V’s architecture.  I’d expect a mechanic to know how an engine works, so a Hyper-V engineer should understand at least the basics of this subject.
  2. Understand that the error dialog presents information from various modules involved in the requested action.
  3. When confused, the Hyper-V-VMMS log is a pretty good place to start digging for treasure.
You can learn more about Windows Server 2012 Hyper-V from the book, Windows Server 2012 Hyper-V Installation And Configuration Guide:

 

Hyper-V Versus vSphere – Comparing Apples With Apples

After reading yet-another-uninformed-pro-vSphere blog post on a tech “news” site, I just have to say something.  Stop.  Please stop.  Today I read on Tech World that Microsoft does not have anything to compare with DRS.  Eh, Dynamic Optimization anybody?

EDIT: The latest last-gasp from a vFanboy was “Hyper-V does not have bare-metal host deployment”.  No it does not; VMM has that feature – and VMM 2012 R2 adds a hell of a lot more.  I bet ESXi doesn’t do bare-metal host deployment either, eh children?

Then there’s the package comparisons.  VMware’s packaging is a nightmare to figure out.  What feature is in what version of vSphere?  I don’t have a friggin clue.  Pricing vSphere makes choosing a phone plan or a health insurance plan look easy.  To be safe, the “journalists” (I reserve real use of that word for a very small subset of the tech news biz) choose Enterprise Plus, the most expensive SKU of vSphere.

How can we compare Hyper-V versus vSphere?

Microsoft SKU VMware SKU Comments Valid Comparison?
Hyper-V Server 2012 ESXi Free Free versus free, hypervisor only, with no guest OS licensing or host management bundled. Yes
Windows Server Hyper-V vSphere suites On the Microsoft side, you have Hyper-V with guest OS licensing bundled.  No central host management (VMM).
On the VMware side you have their hypervisor with no guest OS bundling PLUS central host management solution (vCenter).		 No

The first comparison compares apples with apples.  The second, the one that lazy “journalists”, like those on TechWorld, choose to use.  It’s not a fair comparison.  Microsoft and VMware do not bundle their products in similar packages.  The missing piece from the Microsoft bundle is System Center – Virtual Machine Manager.  VMM is a central host management solution that does pretty much everything vCenter can do, and more.  But you can’t buy VMM on it’s own.  Let’s keep searching options …

Microsoft SKU VMware SKU Comments Valid Comparison?
Core Infrastructure Suite (CIS) or Enrolment for Core Infrastructure (ECI) vSphere suites Windows Server + all of System Center with host and VM licensing.
From VMware, we still only have host + host management licensing.		 No
Core Infrastructure Suite (CIS) or Enrolment for Core Infrastructure (ECI) VMware vCloud Suite Enterprise Windows Server + all of System Center with host and VM licensing.
From VMware, we now have vSphere Enterprise Plus and a boat load of various management SKUs.		 Close, but no

Comparing CIS/ECI to vSphere suites is *giggles* not fair to VMware – who thought I’d ever say that!?!?!?  In the Microsoft CIS/ECI stack you have all of System Center, a complete service delivery, cloud, backup, health monitoring, infrastructure deliver/management, and so on, for a automating data centre.

I don’t know the VMware stack – as I said, I find it completely confusing compared to the simple Microsoft bundling – but the vCloud suite seems to have a tonne of stuff in it. You could compare the $11,495 vCloud Suite Enterprise with $5,959.20 ECI bundle. You should remember that the price of Windows Server Datacenter for licensing your VMs in $4,809 per 2 CPU host.  That means that to run Windows Server VMs on your vSphere, your cost per 2 CPU host has gone up to $16,304.

So what about the apples-to-apples comparisons for the top end?  Then you need to compare:

Microsoft SKU VMware SKU Comments Valid Comparison?
Core Infrastructure Suite (CIS) or Enrolment for Core Infrastructure (ECI) VMware vCloud Suite Enterprise + Windows Server Datacenter Windows Server + all of System Center with host and VM licensing.
From VMware, we now have vSphere Enterprise Plus and a boat load of various management SKUs, and VM licensing.		 Yes

In summary, Mr/Miss “journalist” for future apple-to-apple comparisons you should stick to one of the following:

  • Hyper-V Server versus ESXi Free
  • Microsoft ECI suite versus VMware vCloud Suite Enterprise plus Windows Server Datacenter

Storage Spaces Inside a Virtual Machine Is Not Supported

I’m hooked on Storage Spaces, the mechanism in Windows Server 2012 where we can aggregate non-RAID disks and create thinly provisioned (optional), fault tolerant volumes, just like you’ve been doing on a modern SAN (but Storage Spaces is more flexible, if not as feature rich).

It appears that some like this feature so much that they’ve started to implement it inside of virtual machines:

THIS IS NOT A SUPPORTED CONFIGURATION

Sure, you might see presenters like myself do this in demos.  I make it clear: I only do this because I don’t have the hardware to do Storage Spaces at the physical layer.  Storage Spaces was designed to be created using physical disks … and then you can store your virtual machines on a Storage Space virtual disk.

Why are people implementing Storage Spaces in a production VM?  My primary guess is that they want to aggregate virtual hard disks to create a larger volume.  VHD format files can only expand up to 2040 GB.  OK … that’s the wrong way to go about it!  The correct solution would be one of the following:

  • Deploy Windows Server 2012 Hyper-V and use VHDX files.  They scale out to 64 TB – the maximum size of a VSS snapshot BTW.
  • If you’re stuck on vSphere (2 TB VMDK) or pre-WS2012 Hyper-V (2040 GB VHD) then (I hate saying this …) use a physical disk of some kind until you can upgrade to a scalable hypervisor like WS2012 Hyper-V and convert to the more flexible VHDX.

A second possible excuse is: “I want to create volumes inside a VM”.  Anyone who has spent any time owning a virtualised platform will laugh at this person.  There is a simple rule in our business: 1 volume = 1 virtual hard disk.  It gives us complete flexibility over volume management both at the physical (placement) and virtual (resizing) layer.  If you need an E: volume, hot-add a VHDX to the SCSI controller.  If you need an F: volume, hot-add a VHDX to the SCSI controller.  If you need to expand the G: volume, expand the G: VHDX and then expand the G: volume.

The other reason I expect to hear via comments is “we’re scared of virtual hard disk corruption so we want to RAID the disks in some way using Storage Spaces”.  Where to start?

  • I have never personally witnessed a corrupt virtual hard disk.  When I have heard of such things it’s because people do stupid things with snapshots or differential disks and they deserve what follows.
  • The VHDX format has built-in protection for corruption that can be caused by power loss.
  • DOING STORAGE SPACES INSIDE A VM IS NOT SUPPORTED!  It’s no one’s fault, other than yours, when it misbehaves or breaks.

Please, just start using VHDX format virtual hard disks ASAP.

Hyper-V PowerShell Script Cookbook on TechNet Wiki

If you’ve heard me speak on Windows Server 2012 or Hyper-V recently, then you know that:

  • I did not really do any PowerShell before March of 2012
  • I started then to solve small problems
  • I’m a total convert to the ways of PowerShell because it speeds up work, gives me predictable results (minus my typos), and saves me from those repetitive tasks

Not only will you find PowerShell all over Windows Server 2012 Hyper-V Installation And Configuration Guide, but you’ll find that a new valuable resource has appeared on the TechNet Wiki.  There you will find the “Hyper-V PowerShell Script Cookbook”.

The goal of this site is to shared PowerShell snippets and scripts.  Right now, the categories include:

  • Virtual Machine
  • Virtual Hard Disk
  • Network Virtualization
  • Virtual Switch
  • Additional scripts

As all good scripters know, you first start by searching, then copying/pasting, and then modifying to get the results you want.  Why not start here … and then contribute any new stuff you create!?!?!

Podcast – Talking Windows 8 Client Hyper-V With Run As Radio

I recently recorded a podcast with Richard Campbell of Run As Radio.  The topic was, as you might guess, Hyper-V, but this time we focused on Client Hyper-V, the edition of Microsoft’s hypervisor that comes in Windows 8 Pro/Enterprise.

You can find the podcast episode here, and I would strongly recommend that you scroll through previous episodes too see what else you can learn from this valuable resource.  The podcast has been on my iTunes subscriptions for quite a while now, and teaches me loads while out on road trips for work.

Hey Look–Your Business Is Running On A 10-Year Old Server Operating System (W2003)

April 2003 was such a fine month.  SARS caused every person with a sniffle to think they’d die.  The war in Iraq was coming to an end (!?).  BA and Air France announced the end of supersonic flight.  We suspected that North Korea might be playing with nukes.  Something called iTunes was launched.  I think that was a fad and disappeared quickly (I’m really hoping a digital Indiana Jones reads this in 10,000 years and thinks it’s the truth).  And on April 24th 2003, Microsoft released Windows Server 2003.

Happy 10th birthday Windows Server 2003!  You were a wonderful operating system.  In my first job where I designed/ran a global infrastructure for a corporate, I chose you just 1 month after GA as the basis for the business.  Sure, it was a bit bleeding edge.  Yes, people did question my decision.  But it worked out fantastically.

I don’t drive a 10-year old car.  I wouldn’t want to use a van in a courier business that I can’t get spare parts for.  In business I want to look forward and be competitive & flexible, instead of clinging to what was right 10 years ago.  Makes you wonder why 57% of servers are still running W2003.

End of support is coming on 14/July/2015.  I really don’t want to hear excuses.  The fact is that the end of extended support is coming.  It is time to start planning your migration from the decade-old operating system.  It’s time to move to an OS that is built for the way we work now.  It is time to pressure vendors and suppliers to support an OS that will actually have some level of support.

Microsoft will not change the date that extended support ends.  Stop fooling yourself, and stop listening to people who eat from their own rear-ends.  Security patches stop, and Microsoft Support will stop taking your calls.  You now have approximately 2 years and 3 months to get moving.  And trust me, that time will fly by so don’t procrastinate.

Blogging On The Petri IT Knowledgebase

Update your RSS feeds: I have just signed up with the Petri IT Knowledgebase to write about Microsoft virtualisation.  I will absolutely be continuing to blog on here.  My goal is to do different kinds of topics on both sites.

Yes, I know this site has been kind of quite in the last 2 weeks – I’m snowed under with prepping and delivering presentations for various road shows and events.

You can see part 1 of my newest article on Petri, where I start exploring the various kinds of storage that you can use in a Hyper-V virtual machine.

Passthrough Disks –ARE– Still Supported By WS2012 Hyper-V Live Migration, But With Limits

I recently reported on a new KB article that says:

Changes and improvements in Windows Hyper-V Server 2012 no longer support pass-through disks if Live Migration is to be used.

In other words, Live Migration was allegedly not supporting the use of passthrough disks.

That article was incorrect. 

The story is:

1) Hans Vredevoort told me that found a contradicting blog comment/response by Jeff Woolsey where he stated that Live Migration of VMs with passthrough disks would be supported in what I’ll call legacy scenarios:

Pass through disks are still supported with Windows Server 2012 Hyper-V Live Migration (just like they were with Windows Server 2008 R2 Hyper-V) as long as the migration of a clustered VM and the pass through disk is managed by the cluster. For migrations outside of a cluster, such as:

  • Shared Nothing Live Migration or
  • Using standalone hosts with file on a SMB share (without clustering enabled)

…pass through disks aren’t supported because the pass through disk doesn’t have a way to move between hosts.

That makes total sense.  The passthrough disk (which is not a CSV) has to be passed from one host to another, and only a cluster-managed disk can do this.

Therefore the new scenarios in WS2012 Hyper-V cannot support Live Migration.  Again, that makes total sense:

  • Non-clustered SMB-stored-VM Live Migration – you can’t store a LUN on a file share!
  • Shared-Nothing Live Migration – until you can transport a LUN from one server to another Star-Trek-style, it isn’t happening.

2) Who was correct, Jeff Woolsey or the KB article?  We needed clarity so I reached out to the Hyper-V group in Redmond.  They responded overnight and Jeff was right.  The KB article was … a … bit enthusiastic (I personally loved the message – I’m quite clear in my training that I’ll smack anyone who I find using passthrough disks).  In fact, that KB article has been deleted by Microsoft.

So those of you who are panicking about your passthrough disks, you can calm down now.

…

However …

The advice from everyone who knows anything about Hyper-V is that you should switch to using VHDX files.  This isn’t just me.  Check out any of the Virtual Machine MVPs on this topic.  Read what Didier Van Hoye or Hans Vredevoort (both being the top 2 storage guys in our group IMO) have to say on the topic.

  • VHDX scales out to 64 TB
  • It has new health features to limit corruption
  • It supports 4K sector matching for performance
  • It offers the legacy VHD types, where Fixed will be nearly the same speed as the underlying disk

I heard loads of complaints over the “death” of passthrough disks in the last 7 days.  To you I say, you need to put down your toys of the 2000s. and join us in the 2010s.  We virtualise now for flexibility.  The business demands it, and passthrough disks are not flexible.

The one argument I’ll give some credence to is “I can’t hot expand a VHDX”.  If you are hot expanding LUNs every couple of days then you’ve got other issues, but I too would like this feature.

Anywho, panic over.  You can Live Migrate a VM with passthrough disks as long as both the VM and the passthrough disks are managed by a Hyper-V cluster.  I’m going back to my lazy vacation day now.

Event – Two Day Deep Dive On Windows Server 2012

Microsoft Ireland are running a free 2-day technical deep dive event on Windows Server 2012 on April 22-23 in Dublin.  The agenda is below.

Monday22nd April

9:00 – 9:30         

Registration
   

9:30 – 10:00       

6 month update (L 100)

10:00 – 11:00     

Fundamentals – storage & networking) (L 200)

11:00 – 11:15     

Coffee

11:15 – 13:00     

Storage – iSCSI, Storage Pools, DeDupe (L 300/400)

13:00 – 13:45     

Lunch

13:45 – 15:00     

Networking – DHCP, SMB 3.0 NIC Teams, DA (L 300/400)

15:00 – 15:15     

Coffee

15:15 – 16:60     

Active Directory, inc. DAC (L 300/400)

16:30                   

End

Tuesday23rd

9:30 – 11:00       

Hyper-V (L 300/400)

11:00 – 11:15     

Coffee

11:15 – 13:00     

Virtual Networks (L 300/400)

13:00 – 13:45     

Lunch

13:45 – 15:00     

Mini-MMS

15:00 – 15:15     

Coffee

15:15 – 16:30    

Mini-MMS

16:30                   

End

A large collection of Microsoft speakers appears to be presenting the content.  I’m presenting the 2 highlighted sessions on Hyper-V and storage.

Remember, there are also some regional events that will be discussing the reasons to upgrade to Windows Server 2012 that are starting next week.

vSphere Doesn’t Need Any Security or Bug Fixes

So it seems that the vFanboys are chuckling to themselves today because they saw some bug fixes being released for Windows Server 2012 & Hyper-V.  I hate to burst your vBubble, but it seems that VMware also releases fixes.

For you IT pro wanabees out there, pretty much any decent amount of code is going to have bugs.  You can test all you want, but it’s really only when code gets out into the real world that the product gets truly stressed across scenarios, hardware, drivers, firmwares, and so on.  So, what can we find on the VMware site?  I did a search on ESXi 5.1 and saw this:

image

So VMware’s developers are human too!  Am I making fun of VMware here?  No.  Am I saying vSphere isn’t fit for usage?  No.  I expect to find bug fixes for heavy duty software.

Am I making fun of the fact that you need to install Java to download these fixes?  You bet your ass I am Smile  Seriously?!?!? You need to use the biggest security vulnerability on the planet to download security and bug fixes from VMware?  Damn!

What about the size of these fixes?  The smallest one is 306.1 MB and the largest is 602.4 MB.  Hmm.  You know how vFanboys love to gufaw about having to install the “bloatware” that is Windows Server to get Hyper-V?  The April update rollup for Windows Server that fixes a long list of things is just 45.4 MB. 

I’m thinking the vFanboys in question need to check their facts first.  You know who you are Smile

Technorati Tags: ,,