Month: June 2010

Hyper-V Backups Cause Large Registry Files

I saw this one in my tweet feed this morning.  It’s an important one for anyone doing host level backups of virtual machines on W2008 R2 Hyper-V host servers.  Ben Armstrong has explained how the registry file of the host will grow over time as you back up virtual machines.  This eventually leads to slow boot up times for the host.

Here’s how I understand the description:

  • A VSS backup at the host (W2008 R2) level backs up a VM
  • The backed up copy of each VHD is temporarily mounted by the parent partition for some cleanup work.
  • This mount causes a registry entry to be created.

Imagine doing this with a VSS enabled and optimized backup product like DPM 2010.   Maybe 15-20 VM’s, with an average of 2 VHD’s per VM, per host being backed up every 15 minutes.  That’s a lot of stuff going on.

Ben also warns that the fix, KB982210, is titled incorrectly.  It says Windows Server 2008 when it should say Windows Server 2008 R2.

By the way, this fix is thanks to the way that a number of MS people have embraced social media.  Ben asked people to contact him if they encountered issues with Hyper-V backups.  A number of people raised this issue, and we go this fix.

First Impressions: Windows Server 2008 R2 NLB

I have a customer that is using Windows Network Load Balancing for a fault tolerance web service across two web servers.  They started running W2003 x86 a few years ago and recently decided that they should “upgrade” to W2008 R2 to take advantage of some of the better web hosting features.  Each server has 2 NIC’s.  The first is the normal one we use to log into the servers and manage them.  The second is used purely for the NLB clustered web traffic.

This meant a rebuild of the servers.  For some architectural reasons, it was also decided to build a new NLB cluster.  We would do this one web server at a time.

We rebuilt the first server.  I brought up a new NLB cluster, with just itself as the only member for the moment. We would add the second server when it was rebuilt.  To bring it into production we would:

  • Change the production IP address on the old NLB cluster to a temporary one.
  • Change the temporary IP address on the new NLB cluster to the production on.

Then we could rebuild the second web server and away we go!

Muggins here drew the short straw and I was awake at 06:00 this morning to VPN in, do some prep work and switch the IP addresses to bring the new server into production.  I did that and tested.  The websites would not respond.  I had no idea what was up.  Network monitor showed external traffic coming in on TCP 80 and reaching the server.  I could even see my IP address coming in.

I checked the website bindings which were set to the default of *, that is all assigned IP addresses on the server.  I verified with IPCONFIG that the production IP was live.  I could ping it from other machines and see the traffic in Network Monitor.  I decided I would configure the site in IIS7.5 to just use the NLB cluster IP address.  That’s where issue #1 arose.  I could not select that IP address.  After a quick google I learned that W2008 R2 IIS7.5 cannot pick detect the NLB cluster IP address and load it into the drop down list box.  I had to type it in.

It should be OK now?  I tested.  And no joy.  At this point I had to roll back the changes. The site had been offline for too long.

A few hours later I had the time to start investigating some more.  I used another public IP address with a NAT rule to another internal IP address that I could use on the new NLB cluster.  That would leave the production, old NLB, websites up and running and unaffected by my tests.

I still couldn’t access the site.  I tested the sites from another server in the same VLAN.  I could access the sites from there.  Strange!  This means that I either had a firewall or a routing issue.  It couldn’t be a firewall issue.  The same NAT rule was being used on the new server.  I was simply moving the IP address and we don’t do anything crazy with MAC addresses.  It couldn’t be an ARP cache issue because I could see web traffic actually reaching the server in Network Monitor 3.3.

I scratched my head.  I could route out from the server.  I could surf the web and traceroute out.  Both the server’s management IP and NLB IP are in the same VLAN.  The server management IP had the correct default gateway.  The TCP configuration was identical to the W2003 R2 configuration.

What if … now I was reaching … what if NLB doesn’t route correctly?  What if the NLB NIC’s IP configuration doesn’t pick up the default gateway set up on the management NIC’s IP configuration.  If it was a normal NIC it probably would.  I set up the default gateway on the NLB NIC.  It was identical to the server management NIC configuration.  I got the warning about multiple default gateways on a computer and clicked OK.

Now I tested web site access from an external IP and it worked perfectly.  My conclusion?  You have to configure the default gateway on an NLB NIC if using Network Load Balancing on Windows Server 2008 R2.  Otherwise it will not route correctly to other networks; it should pick up the default gateway from the management NIC but it does not.

Technorati Tags:

Is @VodafoneIreland Censoring Legitimate Internet Content?

I had an appointment early this morning.  I was there early and decided to read the news on my Windows mobile phone.  I’m a reader of the Irish Independent on http://www.independent.ie.  I was reading a couple of pages and then suddenly I could no longer access the site.

I was presented with this error:

HTTP Error 403: The service you requested is restricted.

The service you requested is restricted and not available to your browser.

That was funny because the site was OK 10 seconds before hand and had been since December when I bought the phone.

The page in question is hosted by 10.163.236.214 at http://10.163.236.214/lnk000/=http://imgvip.local/loc/local/home.aspx.  This is not an independent.ie server telling me I cannot get onto their site.  Hardly!  This is a Vodafone Ireland server blocking me from accessing the site for no good reason.

This is different to the usual block which comes from another server in the 10.163 range.  That message blocks access to content for under 18’s (or those who haven’t cleared themselves in a Vodafone Ireland shop).  I can easily bring up that different warning when I try to access Facebook or Twitter.

So what is it Vodafone Ireland.  I know you read my blog.  I get about a dozen hits from you folks every day.  Are you censoring legitimate content?  This is odd, because my cousin (a IT security expert) did suspect that the dodgy performance on your home broadband suggested that there was a transparent proxy being used.  Every Vodafone Ireland employee he and I spoke to denied that there was.  Now I am seeing a block – that means there most definitely is a transparent proxy, at least on your mobile network, and I am left to suspect that there is one on your fixed line broadband.

This censoring of legitimate access to a newspaper site that is critical of the ruling (and some would say corrupt) government is very suspicious. 

Come on, let’s hear it then?  What have you done?  Be very honest because you know I will be when I write to the Independent and let them know about this.

Ben Armstrong’s TechEd Session on Dynamic Memory

The “Virtual PC Guy”, Ben Armstrong, presented at TechEd yesterday on the subject of Dynamic Memory.  I guess you could call it DM’s coming out party.  Ben posted his presentation on his blog.  It won’t surprise me if there are a number of public webcasts in the coming months.

New VMM 2008 R2 Rollup Update

Microsoft has just announced a rollup update for System Center Virtual Machine Manager 2008 R2.  It fixes:

  • A high availability virtual machine with a GPT pass-thru disk is reported is reported as “Unsupported Cluster Configuration.” 
  • When you manage VMware Virtual Center that has distributed switches, the Virtual Machine Manager service (Vmmservice.exe) crashes.
  • After Quick Storage Migration is completed, the Failover Cluster Management MMC snap-in reports that the VM still uses the old CSV volume.
  • If a VM is configured with 3 virtual processors, the SCVMM Admin Console crashes.
  • The settings for the virtual network are incorrect after a Refresh-VMHost cmdlet job is completed.

This update is being released through Windows Update.  That’s totally convenient and I love it.  Microsoft says that “if the Virtual Machine Manager or Virtual Machine Manager Agent service is running when the hotfix rollup package is installed on the Virtual Machine Manager server, you have to restart the computer”.

Technorati Tags: ,,

W2008 R2 SP1 Beta By End of July

Due to another engagement, I couldn’t watch the Muglia keynote from TechEd today.  I missed the expected announcement regarding the beta release of Windows 2008 R2 and Windows 7 Service Pack 1.  We should expect it by the end of July.  For you Hyper-V heads, that’s when we get the first official public glimpse at Dynamic Memory.  For you remote desktop services heads, RemoteFX is in there too.

To be honest, I was hoping the release would be tonight.  But it isn’t 🙁

David Houston RIP

Last week we heard the sad news that David Houston had died.  David was a prominent member in the Irish IT community.  He was one of the founders of the SBS user group and continued to run it with impressive events.  David also spoke at the Windows user group last year.

The funeral will be at 14:00 today in Delgany, Co. Wicklow.