Let\u2019s get something straight.\u00a0 Hyper-V snapshots should only ever be used in lab\/demo environments.\u00a0 If you need a snapshot in production you really should be using DPM 2007 SP1.<\/p>\n
Note (posted 2\/1\/2012): Hyper-V does support snapshots in production, but almost every app you’ll install in a VM doesn’t support any virtualisation snapshot (not VSS\/SAN, but virtualisation).<\/em><\/p>\n Note: CSV doesn\u2019t have a DPM solution yet.\u00a0 DPM v3 will support CSV.<\/em><\/p>\n We don\u2019t have DPM so I only ever use snapshots as a very temporary thing when doing an upgrade within a VM.\u00a0<\/p>\n That\u2019s what I did when I upgraded our Operations Manager VM from OpsMgr 2007 to OpsMgr 2007 R2.\u00a0 I took a snapshot and did the upgrade.\u00a0 I was certain that I had merged the snapshot after doing the upgrade but it appears I didn\u2019t.<\/p>\n Over the weekend our OpsMgr started firing alerts out.\u00a0 They were all network related, e.g. failed heartbeats, inaccessible network devices, websites not responding.\u00a0 I was stunned at the quantity.\u00a0 I was quick to verify there were no outages.\u00a0 These were false alarms.\u00a0 Our network guys investigated.\u00a0 We thought we found a cause but it wasn\u2019t.<\/p>\n I got the noise under control and continued to work on the issue today at the TechEd Europe 2009 conference.\u00a0 I couldn\u2019t find anything.\u00a0 Our network guys did identify that they were losing 3% of pings to the OpsMgr server.\u00a0 That would sure cause the issues we were seeing.\u00a0 I was concerned because our OpsMgr server is a VM.\u00a0 Could the hardware have an issue and could this affect the other VM\u2019s.\u00a0 Some quick tests showed that all hardware and all other VM\u2019s were 100% fine.\u00a0 This problem was limited to our OpsMgr VM.<\/p>\n I decided on this plan:<\/p>\n The reboot did nothing.\u00a0 I attempted a cold migrate.\u00a0 That failed almost as soon as the job started.\u00a0 The VHD\u2019s were locked.\u00a0 I logged into the host and fired up the Hyper-V console to have a look around.\u00a0 That\u2019s when I saw a merge was taking place.\u00a0 A large AVHD was being merged back into the VHD.<\/p>\n What was happening?\u00a0 Remember that when you take a snapshot in Hyper-V it creates a special differential disk called a AVHD.\u00a0 It becomes the place where all writes are stored.\u00a0 The original VHD becomes read only for old data.\u00a0 That becomes pretty slow.\u00a0 I think that it must have degraded performance within the VM so much that it affected the TCP stack of the Windows installation running in the VM.<\/p>\n Eventually the merge completed.\u00a0 It did take a while on our 15K EVA SAN disks.\u00a0 I did the cold migration and fired up the VM.\u00a0 I fired 500 pings at it with 0% loss.\u00a0 Monitoring has raised zero alerts 50 minutes later.\u00a0 It appears to me that I\u2019ve fixed the problem.\u00a0 It appears to me that an overlooked snapshot made my life hell today.<\/p>\n That was my second night at the conference.\u00a0 It wasn\u2019t a beerfest for me.\u00a0 In fact, I worked on this last night too.\u00a0 This one really baffled me.\u00a0 I was sure it was hardware.\u00a0 Turns out it was virtual hardware couple with my feeble brain \ud83d\ude42<\/p>\n","protected":false},"excerpt":{"rendered":" Let\u2019s get something straight.\u00a0 Hyper-V snapshots should only ever be used in lab\/demo environments.\u00a0 If you need a snapshot in production you really should be using DPM 2007 SP1. Note (posted 2\/1\/2012): Hyper-V does support snapshots in production, but almost every app you’ll install in a VM doesn’t support any virtualisation snapshot (not VSS\/SAN, but … Continue reading “How A Hyper-V Snapshot Made My Day Hell”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_uf_show_specific_survey":0,"_uf_disable_surveys":false,"footnotes":""},"categories":[20],"tags":[],"class_list":["post-9967","post","type-post","status-publish","format-standard","hentry","category-hyper-v"],"aioseo_notices":[],"jetpack_featured_media_url":"","amp_enabled":true,"_links":{"self":[{"href":"https:\/\/aidanfinn.com\/index.php?rest_route=\/wp\/v2\/posts\/9967","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/aidanfinn.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/aidanfinn.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/aidanfinn.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/aidanfinn.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=9967"}],"version-history":[{"count":0,"href":"https:\/\/aidanfinn.com\/index.php?rest_route=\/wp\/v2\/posts\/9967\/revisions"}],"wp:attachment":[{"href":"https:\/\/aidanfinn.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=9967"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/aidanfinn.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=9967"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/aidanfinn.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=9967"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}\n