{"id":11399,"date":"2011-07-26T18:35:59","date_gmt":"2011-07-26T18:35:59","guid":{"rendered":"https:\/\/aidanfinn.com\/?p=11399"},"modified":"2011-07-26T18:35:59","modified_gmt":"2011-07-26T18:35:59","slug":"another-hyper-v-implementation-mistake-too-many-csvs","status":"publish","type":"post","link":"https:\/\/aidanfinn.com\/?p=11399","title":{"rendered":"Another Hyper-V Implementation Mistake \u2013Too Many CSVs"},"content":{"rendered":"

In the PowerPoint<\/a> that I posted yesterday, I mentioned that you should not go overboard with creating CSVs (Cluster Shared Volumes).  In the last two weeks, I\u2019ve heard of several people who have.  I\u2019m not going to play blame game.  Let\u2019s dig into the technical side of things and figure out what should be done.<\/p>\n

In Windows Server 2008 Hyper-V clustering, we did not have a shared disk mechanism like CSV.  Every disk in the cluster was single owner\/operator.  Realistically (and required by VMM 2008) we had to have 1 LUN\/cluster disk for each VM.<\/p>\n

That went away with CSV in Windows Server 2008 R2.  We can size our storage (IOPS from MAP) and plan our storage (DR replication, backup policy, fault tolerance) accordingly.  The result is you can have lots of VMs and virtual hard disks (VHDs) on a single LUN.  But for some reason, some people are still putting 1 VM, and even 1 VHD, on a CSV.<\/p>\n

An example: someone is worried about disk performance and they spread the VHDs of a single VM across 3 CSVs on the SAN.  What does that gain them?  In reality: nothing.  It actually is a negative.  Let\u2019s look at the first issue:<\/p>\n

SAN Disk Grouping is not like Your Daddy\u2019s Server Storage<\/span><\/strong><\/p>\n

If you read some of the product guidance on big software publisher\u2019s support site, you can tell that there is still some confusion out there.  I\u2019m going to use HP EVA lingo because it\u2019s what I know.<\/p>\n

If I had a server with internal disks, and wanted to create three RAID 10 LUNs, then I would need 6 disks.<\/p>\n

\"image\"<\/a><\/p>\n

The first pair would be grouped together to make LUN1 at a desired RAID level.  The second pair would be grouped together to make the second LUN, and so on.  This means that LUN1 is on a completely separate set of spindles to LUN2 and LUN3.  They may or may not share a storage controller.<\/p>\n

A lot of software documentation assumes that this is the sort of storage that you\u2019ll be using.  But that\u2019s not the case with a cluster with a hardware SAN. You need to use the storage it provides, and it\u2019s usually nothing like the storage in a server.<\/p>\n

By the way, I\u2019m really happy that Hans Vredevoort is away on vacation and probably will miss this post.  He\u2019d pick it to shreds \"Smile\"<\/em><\/p>\n

Things are kind of reversed.  You start off by creating a disk group (HP lingo!)  This is a set of disks that will work as a team, and there is often a minimum number required.<\/p>\n

\"image\"<\/a><\/p>\n

From there you will create a virtual disk (not a VHD \u2013 it\u2019s HP lingo for a LUN in this type of environment).  This is the LUN that you want to create your CSV volume on.  The interesting thing is that each virtual disk in the disk group spans every disk in the disk group.  How that spanning is done depends on the desired RAID level.  RAID 10 will stripe using pairs of disks, and RAID5 will stripe using all of the disks.  That gives you the usual expected performance hit\/benefits of those RAID levels and the expected available amount of data.<\/p>\n

In the below, you can see two virtual disks (LUNs) have been created in the disk group.  The benefit of this approach is that the virtual disks can benefit by having many more spindles to use.  The sales pitch is that you are getting much better performance than the alternative server internal storage.  Compare LUN1 from above (2 spindles) with vDisk1 below (6 spindles).  More spindles = more speed.<\/p>\n

I did say it was sales pitch.  You\u2019ve got other factors like SAN latency, controller cache\/latency, vDisks competing for disk I\/O, etc.<\/em> But most often, the sales pitch holds fairly true.<\/em><\/p>\n

\"image\"<\/a><\/p>\n

If you think about it, a CSV spread across a lot of disk spindles will have a lot of horsepower.  It should provide excellent storage performance for a VM with multiple VHDs.<\/p>\n

A MAP assessment is critical.  I\u2019ve also pointed out in that PowerPoint<\/a> that customers\/implementers are not doing this.  This is the only true way to plan storage and decide between VHD or passthrough disk.  Gut feeling, \u201cexperience\u201d, \u201cknowledge of your network\u201d are a bunch of BS.  If I hear someone saying \u201cI just know I need multiple physical disks or passthrough disks\u201d then my BS-ometer starts sending alerts to OpsMgr \u2013 can anyone write that management pack for me?<\/em><\/p>\n

Long story short: a CSV on a SAN with this type of storage offers a lot of I\/O horsepower.  Don\u2019t think old school because that\u2019s how you\u2019ve always thought.  Run a MAP assessment to figure out what you really need.<\/p>\n

Persistent Reservations<\/span><\/strong><\/p>\n

Windows Server 2008 and 2008 R2 Failover Clustering use iSCSI3 persistent reservations (PRs) to access storage.  Each SAN solution has a limit on how many PRs they can support.  You can roughly calculate what you need using:<\/p>\n

PRs = Number of Hosts * Number of Storage * Channels per Host Number of CSVs<\/p>\n

Let\u2019s do an example.  We have 2 hosts, with 2 iSCSI connections each, with 4 CSVs.  That works out as:<\/p>\n

2 [hosts] * 2 [channels] * 4 [CSVs] = 16 PRs<\/p>\n

OK; Things get more complicated with some storage solutions, especially modular ones.  Here you really need to consult an expert<\/em> (and I don\u2019t mean Honest Bob who once sold you a couple of PCs at a nice price).  The key piece may end up being the number of storage channels.  For example, each host may have 2 iSCSI channels, but it maintains connections to each module in the SAN.<\/p>\n

Here\u2019s another example.  There is an iSCSI SAN with 2 storage modules.  Once again, we have 2 hosts, with 2 iSCSI connections each, with 4 CSVs.  This now works out as:<\/p>\n

2 [hosts] * 4 [channels \u2013> 2 modules * 2 iSCSI connections] * 4 [CSVs] = 32 PRs<\/p>\n

Add 2 more storage modules and double the number of CSVs to 8 and suddenly:<\/p>\n

2 [hosts] * 8 [channels \u2013> 4 modules * 2 iSCSI connections] * 8 [CSVs] = 128 PRs<\/p>\n

Your storage solution may actually calculate PRs using a formula with higher demands.  But the question is: how many PRs can your storage solution handle?  Deploy too many CSVs and\/or storage modules and you may find that you have disks disappearing from your cluster.  And that leads to very bad circumstances.<\/p>\n

You may find that a storage firmware update increases the number of required PRs.  But eventually you reach a limit that is set by the storage manufacturer.  They obviously cripple the firmware to create a reason to buy the next higher up model.  But that\u2019s not something you want to hear after spending \u20ac50K or \u20ac100K on a new SAN.<\/p>\n

They way to limit your PR requirement is to deploy only the CSVs you need.<\/p>\n

Undoing The Damage<\/strong><\/span><\/p>\n

If you find yourself in the situation with way too many CSVs then you can use SCVMM Quick Storage Migration to move VMs onto fewer, larger CSVs, and then remove the empty CSVs.<\/p>\n

Recommendations<\/span><\/strong><\/p>\n

Slow down to hurry up.  You MUST<\/span><\/em><\/strong><\/span> run an assessment of your pre-virtual environment to understand what storage you buy.  You also use this data as a factor for planning CSV design and virtual machine\/VHD placement.  Like my old woodwork teacher used to say: \u201cmeasure twice and cut once\u201d.<\/span><\/span><\/p>\n

Take that performance requirement information and combine it with backup policy (1 CSV backup policy = 1 or more CSVs, 2 CSV backup policies = 2 or more CSVs, etc), fault tolerance (place clustered or load balanced VMs on different CSVs), and DR policy (different storage level VM replication policies requires different CSVs).<\/p>\n

Technorati Tags: Hyper-V<\/a>,Failover Clustering<\/a>,Virtualisation<\/a>,Windows Server 2008 R2<\/a>,Backup<\/a><\/div>\n","protected":false},"excerpt":{"rendered":"

In the PowerPoint that I posted yesterday, I mentioned that you should not go overboard with creating CSVs (Cluster Shared Volumes).  In the last two weeks, I\u2019ve heard of several people who have.  I\u2019m not going to play blame game.  Let\u2019s dig into the technical side of things and figure out what should be done. … Continue reading “Another Hyper-V Implementation Mistake \u2013Too Many CSVs”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_uf_show_specific_survey":0,"_uf_disable_surveys":false,"footnotes":""},"categories":[20],"tags":[55,63,181,195,117],"class_list":["post-11399","post","type-post","status-publish","format-standard","hentry","category-hyper-v","tag-backup","tag-failover-clustering","tag-hyper-v","tag-virtualisation","tag-windows-server-2008-r2"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t