I love it when I read about someone saying \u201cmy virtualisation solution support more processors and more memory in a host than your one\u201d.\u00a0 It\u2019s like arguing over who\u2019s a better captain: Kirk or Picard? By the way, it\u2019s Janeway.<\/p>\n
In my experience I\u2019ve yet to see a host with more than 200 GB RAM or 4 sockets (physical processors).\u00a0 And they\u2019ve been the rare ones.\u00a0 Most have been 1 or 2 CPUs, with 32-48GB RAM, and often less than that.<\/p>\n
For me, sizing a host comes down to a few things that need to be balanced:<\/p>\n
Purchase cost is only the starting point of the cost of own your new shiny piece of kit.\u00a0 These things can cost\u00a0as much\u00a0to run over 3 years than to purchase.\u00a0 But I hardly ever hear of anyone trying to figure that cost out, ask about it, or include it in their budgeting.\u00a0 That \u20ac10,000 server can cost you a total of \u20ac20,000 over 3 years.\u00a0 Build a a 10 node cluster and the numbers get big pretty quick.\u00a0 The ownership cost is complicated, but some of the big elements are host licensing, rack space, and the ever increasing cost of electricity.<\/p>\n
OK, let\u2019s assume we\u2019ve done a sizing process and we need host fault tolerance with 40+ vCPUs and 400GB of RAM for the VMs.\u00a0 Storage will be courtesy of a 10 GB iSCSI SAN.\u00a0 How do you size those hosts?\u00a0 Do you get 2 big old beasts with enough capacity for all the VMs?\u00a0 Or do you get lots of machines that are stocked full of 4 GB DIMMs?<\/p>\n
Things got a little more complicated in the last 12-18 months because hardware manufacturers are giving us machines that can support 24, 32 or more DIMMs in a single server.\u00a0 One machine (a HP DL585 G7) can take 4 * 12 core AMD processors (48 cores \u2013 giving you enough vCPU capacity to exceed the maximum support limits of Hyper-V with the new W2008 R2 SP1 ratio of 12:1 vCPUs to cores) and 512 GB RAM by using 16 GB DIMMs.\u00a0 But here\u2019s the catch: how much does that beastie cost?\u00a0 A high end CPU can cost around \u20ac1,200.\u00a0 And pricing for DIMMs is not linear.\u00a0 In other words a 16 GB DIMM costs a good deal more than 4 * 4 GB DIMMs.<\/p>\n
On the plus side with the big beastie, you are able to minimise your power consumption.\u00a0 If carbon footprint is your primary objective then this is your puppy! Or is it?\u00a0 Doesn’t fewer servers equal less power?<\/p>\n
OK, so the big beast is expensive.\u00a0 What about going for something that uses 4 GB DIMMs? \u00a0Typically that\u2019ll mean a 2 CPU server with 96 GB of RAM (96 GB is the new 32 GB).\u00a0<\/p>\n
This does mean that you\u2019re using more economic components.\u00a0 But this has an interesting effect on power costs.\u00a0 They go up.\u00a0 You\u2019re using more CPUs, more power supplies, more rack space, more networking, and the cost goes on and on.<\/p>\n
So where is the sweet spot?\u00a0 I\u2019ve done some very rough sums using the Irish retail prices of HP servers and components, in combination with the HP power calculator with Irish electricity prices.\u00a0 I\u2019ve taken the hardware costs, the power costs over 3 years, and created a total cost of owning a host server solution.\u00a0 And then I took the above requirements to size and price up 4 different solutions using the big iron servers, and the budget spec servers, and a couple of points between.<\/p>\n
Host Spec<\/strong><\/td>\n| Power Cost<\/strong><\/td>\n | Bid Price Hardware (80%)<\/strong><\/td>\n | Total Cost (3 Years)<\/strong><\/td>\n<\/tr>\n | 3 * DL385 G7, 2 * 12 Core, 256 GB RAM<\/td>\n | \u20ac21,629<\/td>\n | \u20ac38,027.40<\/td>\n | \u20ac59,656.40<\/td>\n<\/tr>\n | 4 * DL385 G7, 2 * 8 Core, 192 GB RAM<\/em><\/td>\n | \u20ac22,109<\/em><\/td>\n | \u20ac32,836.20<\/em><\/td>\n | \u20ac54,945.2<\/em><\/td>\n<\/tr>\n | 6 * DL385 G7, 2 * 8 Core, 96 GB RAM<\/td>\n | \u20ac31,567<\/td>\n | \u20ac39,607.80<\/td>\n | \u20ac71,174.80<\/td>\n<\/tr>\n | 2 * DL585 G7, 4 * 12 Core, 512 GB RAM<\/td>\n | \u20ac27,698<\/td>\n | \u20ac66,743.60<\/td>\n | \u20ac94,441.60<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n | A few notes on the pricing first.\u00a0 I took retail pricing from the HP Ireland site and assumed a 20% discount for the bid price.\u00a0 That is pretty conservative.\u00a0 The power costs used Irish retail power rates (all I had available to me).\u00a0 I did not include rack space costs, or network costs (more servers equals more of those, thus driving up prices).\u00a0 Each server had additional CPUs (fastest available), an extra dual port 1GB NIC, an extra dual port 10 GB NIC (iSCSI), and 2 * 300 GB SAS drives.<\/p>\n So what was the result?\u00a0 The big iron DL585 boxes were not the cheapest to power.\u00a0 In fact, they came in third.\u00a0 I was a little surprised by this.\u00a0 I guess all those 16 GB DIMMs and 4 CPUs require a lot of cooling.\u00a0 There was no low power 16 GB DIMM; I used low power 4 GB and 8 GB DIMMs in the 2 middle specifications.<\/p>\n The DL385 G7 seemed to be the way to go then.\u00a0 I picked out models that came with the fastest of the AMD CPUs that were available.\u00a0 I then tweaked the choice of memory, this increasing\/decreasing the number of hosts required for the VM RAM load, and further tweaked the CPU cores that were used (8 or 12) to match requirements.<\/p>\n The \u201cbudget\u201d hardware purchase using 4 GB DIMMs came with a sting in the tail.\u00a0 It was the most expensive solution to power (6 servers instead of 2, 3 or 4), possible because it requires 6 servers.\u00a0 The purchase price was not actually budget at all; it was the second most expensive.<\/p>\n OK, the DL385 G7 is a virtualisation server.\u00a0 Why not spec it according to the maximums using 12 core CPUs and 16 GB DIMMs.\u00a0 This gave me a 3 node cluster.\u00a0 This was the cheapest solution to power, which the greener computing fans will be happy to hear.\u00a0 The purchase price was the second lowest which is good news.\u00a0 But over 3 years the total cost for this solution came in second.\u00a0 Maybe it would do better in a company where servers stay in production for longer.\u00a0 But virtualisation makes it easier to change hardware every 3 years and newer hardware tends to be cheaper to power and offer greater density.<\/p>\n Finally I found the sweet spot.\u00a0 I used the DL385 G7, loaded it with 8 Core CPUs and fully populated it with 8 GB DIMMs, giving me 192 GB RAM per host.\u00a0 This 4 node cluster came with the lowest total purchase price.\u00a0 The CPU switch and the change from 16 GB to 8 GB DIMMs made a huge dent, despite requiring an extra chassis.\u00a0 The power cost was the second highest, but not by much.<\/p>\n So what do I make of all this?\u00a0 I say it in the book, and I find myself saying it several times a week when talking to people.\u00a0 Your business and technology requirements should drive every decision you make.\u00a0 If you work for a company that must have a greener image then you\u2019ll pay a little extra for the solution with the smallest footprint.\u00a0 If you\u2019re concerned about rack space then you\u2019ll take the solution that requires the least number of Us.\u00a0 If you are worried about fault tolerance then you\u2019ll increase the cluster size to spread the load a little more.\u00a0 In my example, it appears that the sweet spot is to use a solution somewhere between the extremes, but with regular server models.<\/p>\n My advice to you is to open up Excel, get the various specifications, get the costs, and use a manufacturer\u2019s power calculator to figure you what this stuff will cost you to power.\u00a0 You\u2019ll probably need someone from Accounts to give you rack space, power, network, etc, costs \u2013 or help you calculate them.\u00a0 Don\u2019t just pick out some arbitrary specification.\u00a0 And to complicate things: bid pricing (which you should be getting) will always change the equation, as will the inevitable price\/model changes, over the following 3 years.\u00a0 And try to do other memory configurations that I haven\u2019t done.\u00a0 There might be more possibilities that I haven\u2019t calculated.<\/p>\n","protected":false},"excerpt":{"rendered":" I love it when I read about someone saying \u201cmy virtualisation solution support more processors and more memory in a host than your one\u201d.\u00a0 It\u2019s like arguing over who\u2019s a better captain: Kirk or Picard? By the way, it\u2019s Janeway. In my experience I\u2019ve yet to see a host with more than 200 GB RAM … Continue reading “Host Sizing Versus Total Cost of Ownership”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_uf_show_specific_survey":0,"_uf_disable_surveys":false,"footnotes":""},"categories":[1],"tags":[64,181,195],"class_list":["post-11048","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-hp","tag-hyper-v","tag-virtualisation"],"aioseo_notices":[],"jetpack_featured_media_url":"","amp_enabled":true,"_links":{"self":[{"href":"https:\/\/aidanfinn.com\/index.php?rest_route=\/wp\/v2\/posts\/11048","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/aidanfinn.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/aidanfinn.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/aidanfinn.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/aidanfinn.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=11048"}],"version-history":[{"count":0,"href":"https:\/\/aidanfinn.com\/index.php?rest_route=\/wp\/v2\/posts\/11048\/revisions"}],"wp:attachment":[{"href":"https:\/\/aidanfinn.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=11048"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/aidanfinn.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=11048"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/aidanfinn.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=11048"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}} |