HP ProLiant Gen8 Servers Launched

HP has launched their Gen8 (not G8) line of ProLiant servers.  The machine I was most interested in was the DL380p Gen8 because that’s the model I’d most encounter in virtualisation.  Some highlights:

  • 2 CPU sockets handling up to 8 cores each
  • 24 DIMM slots (requires HP SmartMemory for warrantee and performance) with a maximum of 768 GB (!!!) RAM using 32 GB RDIMM.
  • Choice between 4 * 1 GbE or 2 * 10 GbE NICs on board.
  • iLO 4

On board management has taken a bit of a leap forward:

  • HP iLO: The HP iLO management processor is the core foundation for HP iLO Management Engine. HP iLO for HP ProLiant servers simplify server setup, engage health monitoring, power and thermal control, and promote remote administration for HP ProLiant ML, DL, BL and SL servers. Furthermore with the new HP iLO is the ability to access, deploy, and manage your server anytime from anywhere with your Smartphone device.
  • HP Agentless Management: With HP iLO Management Engine in every HP ProLiant Gen8 server, the base hardware monitoring and alerting capability is built into the system (running on the HP iLO chipset) and starts working the moment that a power cord and an Ethernet cable is connected to the server.
  • HP Active Health System: HP Active Health System is an essential component of the HP iLO Management Engine. It provides: Diagnostics tools/scanners wrapped into one; Always on, continuous monitoring for increased stability and shorter downtimes; Rich configuration history; Health and service alerts; Easy export and upload to Service and Support.
  • HP Intelligent Provisioning (previously known as SmartStart): HP Intelligent Provisioning offers the ability for out-of-the box single-server deployment and configuration without the need for media”.

On the blade front, there is a new BL460c Gen8:

  • 2 CPU sockets handling up to 8 cores each
  • 16 DIMM slots (requires HP SmartMemory for warrantee and performance) with a maximum of 512 GB RAM using 32 GB RDIMM.
  • One (1) HP FlexFabric 10Gb 2-port 554FLB FlexibleLOM
  • iLO 4

There’s crazy big scalability in each host if you can justify it.  To counter that you have the “too many eggs in one basket” argument.  I wonder how much a 32 GB SmartMemory DIMM costs Smile  To reach the densities that this hardware can offer, you will absolutely need to install the very best of networking such as 10 GbE.  I’d even start wondering about InfiniBand!

Host Sizing Versus Total Cost of Ownership

I love it when I read about someone saying “my virtualisation solution support more processors and more memory in a host than your one”.  It’s like arguing over who’s a better captain: Kirk or Picard? By the way, it’s Janeway.

In my experience I’ve yet to see a host with more than 200 GB RAM or 4 sockets (physical processors).  And they’ve been the rare ones.  Most have been 1 or 2 CPUs, with 32-48GB RAM, and often less than that.

For me, sizing a host comes down to a few things that need to be balanced:

  1. How much physical resource (such as RAM, IOPS, storage bandwidth, fault tolerance, CPU, etc) do I need?  An assessment or a proof of concept will help with this.  Failing that, do some googling for a reference architecture (which will give you a guesstimate).
  2. What will this stuff cost me to buy?  Too many people get caught up on this one.  I’ll come back to this in a moment.
  3. What will this stuff cost me to own?  Ah, the forgotten element of the equation!

Purchase cost is only the starting point of the cost of own your new shiny piece of kit.  These things can cost as much to run over 3 years than to purchase.  But I hardly ever hear of anyone trying to figure that cost out, ask about it, or include it in their budgeting.  That €10,000 server can cost you a total of €20,000 over 3 years.  Build a a 10 node cluster and the numbers get big pretty quick.  The ownership cost is complicated, but some of the big elements are host licensing, rack space, and the ever increasing cost of electricity.

OK, let’s assume we’ve done a sizing process and we need host fault tolerance with 40+ vCPUs and 400GB of RAM for the VMs.  Storage will be courtesy of a 10 GB iSCSI SAN.  How do you size those hosts?  Do you get 2 big old beasts with enough capacity for all the VMs?  Or do you get lots of machines that are stocked full of 4 GB DIMMs?

Things got a little more complicated in the last 12-18 months because hardware manufacturers are giving us machines that can support 24, 32 or more DIMMs in a single server.  One machine (a HP DL585 G7) can take 4 * 12 core AMD processors (48 cores – giving you enough vCPU capacity to exceed the maximum support limits of Hyper-V with the new W2008 R2 SP1 ratio of 12:1 vCPUs to cores) and 512 GB RAM by using 16 GB DIMMs.  But here’s the catch: how much does that beastie cost?  A high end CPU can cost around €1,200.  And pricing for DIMMs is not linear.  In other words a 16 GB DIMM costs a good deal more than 4 * 4 GB DIMMs.

On the plus side with the big beastie, you are able to minimise your power consumption.  If carbon footprint is your primary objective then this is your puppy! Or is it?  Doesn’t fewer servers equal less power?

OK, so the big beast is expensive.  What about going for something that uses 4 GB DIMMs?  Typically that’ll mean a 2 CPU server with 96 GB of RAM (96 GB is the new 32 GB). 

This does mean that you’re using more economic components.  But this has an interesting effect on power costs.  They go up.  You’re using more CPUs, more power supplies, more rack space, more networking, and the cost goes on and on.

So where is the sweet spot?  I’ve done some very rough sums using the Irish retail prices of HP servers and components, in combination with the HP power calculator with Irish electricity prices.  I’ve taken the hardware costs, the power costs over 3 years, and created a total cost of owning a host server solution.  And then I took the above requirements to size and price up 4 different solutions using the big iron servers, and the budget spec servers, and a couple of points between.

Host Spec Power Cost Bid Price Hardware (80%) Total Cost (3 Years)
3 * DL385 G7, 2 * 12 Core, 256 GB RAM €21,629 €38,027.40 €59,656.40
4 * DL385 G7, 2 * 8 Core, 192 GB RAM €22,109 €32,836.20 €54,945.2
6 * DL385 G7, 2 * 8 Core, 96 GB RAM €31,567 €39,607.80 €71,174.80
2 * DL585 G7, 4 * 12 Core, 512 GB RAM €27,698 €66,743.60 €94,441.60

A few notes on the pricing first.  I took retail pricing from the HP Ireland site and assumed a 20% discount for the bid price.  That is pretty conservative.  The power costs used Irish retail power rates (all I had available to me).  I did not include rack space costs, or network costs (more servers equals more of those, thus driving up prices).  Each server had additional CPUs (fastest available), an extra dual port 1GB NIC, an extra dual port 10 GB NIC (iSCSI), and 2 * 300 GB SAS drives.

So what was the result?  The big iron DL585 boxes were not the cheapest to power.  In fact, they came in third.  I was a little surprised by this.  I guess all those 16 GB DIMMs and 4 CPUs require a lot of cooling.  There was no low power 16 GB DIMM; I used low power 4 GB and 8 GB DIMMs in the 2 middle specifications.

The DL385 G7 seemed to be the way to go then.  I picked out models that came with the fastest of the AMD CPUs that were available.  I then tweaked the choice of memory, this increasing/decreasing the number of hosts required for the VM RAM load, and further tweaked the CPU cores that were used (8 or 12) to match requirements.

The “budget” hardware purchase using 4 GB DIMMs came with a sting in the tail.  It was the most expensive solution to power (6 servers instead of 2, 3 or 4), possible because it requires 6 servers.  The purchase price was not actually budget at all; it was the second most expensive.

OK, the DL385 G7 is a virtualisation server.  Why not spec it according to the maximums using 12 core CPUs and 16 GB DIMMs.  This gave me a 3 node cluster.  This was the cheapest solution to power, which the greener computing fans will be happy to hear.  The purchase price was the second lowest which is good news.  But over 3 years the total cost for this solution came in second.  Maybe it would do better in a company where servers stay in production for longer.  But virtualisation makes it easier to change hardware every 3 years and newer hardware tends to be cheaper to power and offer greater density.

Finally I found the sweet spot.  I used the DL385 G7, loaded it with 8 Core CPUs and fully populated it with 8 GB DIMMs, giving me 192 GB RAM per host.  This 4 node cluster came with the lowest total purchase price.  The CPU switch and the change from 16 GB to 8 GB DIMMs made a huge dent, despite requiring an extra chassis.  The power cost was the second highest, but not by much.

So what do I make of all this?  I say it in the book, and I find myself saying it several times a week when talking to people.  Your business and technology requirements should drive every decision you make.  If you work for a company that must have a greener image then you’ll pay a little extra for the solution with the smallest footprint.  If you’re concerned about rack space then you’ll take the solution that requires the least number of Us.  If you are worried about fault tolerance then you’ll increase the cluster size to spread the load a little more.  In my example, it appears that the sweet spot is to use a solution somewhere between the extremes, but with regular server models.

My advice to you is to open up Excel, get the various specifications, get the costs, and use a manufacturer’s power calculator to figure you what this stuff will cost you to power.  You’ll probably need someone from Accounts to give you rack space, power, network, etc, costs – or help you calculate them.  Don’t just pick out some arbitrary specification.  And to complicate things: bid pricing (which you should be getting) will always change the equation, as will the inevitable price/model changes, over the following 3 years.  And try to do other memory configurations that I haven’t done.  There might be more possibilities that I haven’t calculated.

HP P4000 LeftHand SAN/iQ 9.0 Adds CSV Hardware VSS Provider Support

You may know that HP and Microsoft have formed a virtualisation alliance around Hyper-V.  One of HP’s key pieces in the puzzle the the iSCSI SAN formerly known as LeftHand, the P4000 series.

Cluster Shared Volume (CSV) can be backed up using a software VSS provider (i.e. Windows VSS) but this is slow.  When using DPM 2010, it’s recommended to use serialised backups.  If your hardware vendor support it, you can use their hardware VSS provider to take a snapshot in the SAN and then DPM (or whatever backup product) will use that feature for the backup.

Now back to the P4000.  Up until recently, the HP supplied DSM for MPIO was version 8.5.  The SAN/iQ software on the P4000 was also at 8.5.  Lots of people were using the 8.5 hardware VSS provider in the SAN to backup CSVs.  It seems that this was unsupported. by HP (nothing to do with MS).  In fact, it can even cause disk deadlock in a Hyper-V cluster, and lead to 0x0000009E blue screens of death (BSOD) on cluster hosts.  And that’s just the start of it!

HP did release DSM 9.0 and SAN/iQ 9.0 recently for the P4000.  These add support for using the hardware VSS provider for backing up a CSV.


So the SAN/iQ 9.0 release docs say that previous versions of SAN/iQ supported CSVs.  However, the Application Snashot Feature (hardware VSS provider/backup application) of the 8.5 release could not support quiecsed snapshots of CSVs.  In other words, it wasn’t supported to use DPM (or anything else) to perform a storage/host level backup of LeftHand with SAN/iQ 8.5 using the HP hardware VSS provider.  It is supported with v9.0.

HP CloudStart

HP has launched a new “cloud” bundle.  It appears to be based on ProLiant and Integrity blade servers to allow you to build a private cloud.  It comes with the usual options such as Virtual Connects with Flex-10.  The bundle can include VMware or Hyper-V for ProLiants.  HPs own system will be used for Integrity servers.  So far, it just sounds like any old server/storage kit.  Where’s the cloud?  That comes with a software product called Cloud Service Automation.  There’s little info on it that I can find quickly.  I guess it’s some virtualization agnostic job engine for automating the deployment of resources, etc.

The suite is available in Asia and will slowly be made available around the rest of the world by the end of the year.

HP DL380/DL385 G7 – Intel or AMD?

I’ve just been looking at the new 2U servers from HP. 

HP DL385 G7 (AMD) – The cheapest of the 2 CPU offerings

  • HP ProLiant DL385 G7 6174 2P 16GB-R P410i/1GB FBWC Hot Plug SFF 750W RPS IC Sr
  • AMD Opteron™ Model 6174 (12 core, 2.2 GHz, 12MB L3, 80W)
  • 2 CPU
  • 16 GB RAM
  • Smart Array P410i/1GB FBWC Controller
  • €5265 excluding VAT

HP DL380 G7 (Intel) – This is the cheapest 6-core Intel box

  • HP ProLiant DL380 G7 X5650 2P 12GB-R P410i/1GB FBWC 8 SFF 750W RPS IC Server
  • Intel® Xeon® X5650 (6 cores, 2.66 GHz, 12MB L3, 95W)
  • 2 CPU
  • 12 GB RAM
  • Smart Array P410i/1GB FBWC Controller
  • €5180 excluding VAT

You can get more computing power from the AMD server: 16 Cores and 16GB of base RAM compared to 12 cores and 12GB RAM in the Intel.  That could be the difference between an additional number of virtual machines, anywhere between 1 high end VM and 32 low end VM’s (based purely on CPU – additional memory would be required).

There is a 12 core AMD CPU model listed but it does not have a list price.  2 CPU’s would give it 24 cores!  That’s 48 Exchange 2010 VM vCPU’s or 24 SharePoint 2010 VM vCPU’s.  With the default 8:1 ratio that is 192 vCPU’s!

You should consider more than just hardware and scalability too.  If you are doing Windows 2008 R2 Datacenter per-CPU licensing for the parent partition (and VM’s) and System Center Management Suite (datacenter) per processor licensing for System Center then you save money with bigger CPU’s.  The more VM’s you can put on a host with bigger CPU’s –> fewer hosts & fewer CPU’s –> less licensing –> less space used –> less host management –> etcetera.

Right now, if I was doing a green field Hyper-V deployment then I would most certainly go with AMD CPU’s.  There is better value with their offering.

If you are expanding an existing Hyper-V deployment then you should stick with all-Intel or all-AMD so that VM’s can move between the hosts.

Technorati Tags: ,,

MMS 2010 Keynote: Server Management

I’m tuning into the live webcast of today’s (there’s another tomorrow) Microsoft Management Summit 2010 keynote featuring server management.  I’ll be doing my best to blog about new stuff as it happens.

System Center Service Manager 2010 is announced as RTM.  Sorry dudes!  YEARS of work (and rework) and I thought you’d get more of a launch than that.

Jeez, an hour later and I’ve not got much more to report.  There’s a lot of talk about cloud (nothing new) and a lot of talk about old concepts (using System Center to do more, and more engineering rather than operations).

EDIT: Someone on Twitter counted the number of times “cloud” was mentioned.  The final count was 83.  Cloud OD.

The next generation of System Center data center is based on lessons from Azure and Bing.  Edwin Yuen hits the stage.  Now we’re cooking!

VMM v.Next

It looks quite different!  It has the cleaner v.Next interface rather than the Outlook 2007 one we are used to.  Server application virtualization, SQL models and MSDeploy (IIS) packages live in the library.  The template model is evolved to a service template spanning multiple servers or tiers.  We see a demo of a 3 tier application.  You can drop OS templates (that we know) and “Server App-V”/MSDeploy packages which we can drop into the model.  You can say that you want X numbers of server in a tier in the model.  You can tier your storage to standard or high performance.  So you’ve got X variations of servers made from a few Server App-V images and OS templates. 

Seriously – I could use this right now.  I have recurring deployments that I could model like this.

You can integrate with WSUS and perform a patching compliance report based on the VHD in the library!  You can then remediate this image in the library.  Now – VMM knows which VMM managed VM’s need to be updated!  You don’t need to patch the running VM OS.   You can <Update Service>, to replace the running OS, while keeping the Server App-V package.

Operations Manager & Azure

How you can monitor Azure and on-premises for seamless application monitoring using OpsMgr 2007 R2.  We see a distributed application containing traditional monitored items (including databases and web watchers) and an Azure presence.  OpsMgr integrates into Azure using a soon-to-be-released (“later this year sometime”) management pack to gather performance information.  A task is there to add new web role instances in Azure.  Nice and simple! 

Deployment of more Azure instances is based on real (synthetic transaction monitoring) measured performance data.  Expansion (or withdrawal) of new instances can be easily done through the same monitoring interface based in your site.

That’s the end.  Really only had good content in the last 22 minutes of a 82 minute keynote.  A quite short post compared to what I would do at an MS Ireland event lasting the same time (see last week for a 3 hour session).

HP G7 Servers

HP has announced the G7 generation of servers.  There seems to be 3 models to start with, all Opteron based.  I’m looking at a video for the DL 385 G7.  Wow!  It takes up to 2 * 12 Core Opteron processors, 12MB L3 cache, with 4 channels of DDR3 memory.  You get double the RAM capacity and 66% throughput compared to the same G6 machine.  24 Dimms with 256GB total capacity.  You have the choice of the usual ECC RAM, cheaper unbuffered RAM and low power RAM.  There are 4 NIC’s built into it – aimed squarely at, but hardly limited to, virtualisation.

The cheaper RAM might be an option for clustered virtualisation were power reduction isn’t the primary goal.  Board fails?  OK – the VM fails over to another host.

New storage controllers with battery and flash backed caches are included.  There is an extra PCIx slot giving you 6 generation 2 slots.

An SD card reader is built in, handy for OEM provided ESXi or Hyper-V Server installations that boot from a card.  Interesting – HP always used to use Compact Flash.

ILO3 promises much better (3 times faster) remote administration.  The big flaw in ILO2 was remote media.  That’s improved now.  You can script power usage for low power (UPS) scenarios.  There is also a power measurement feature now.

There are 8 built in SFF slots.  You can add another cage to have a total of 16.  Instead of 8 SFF, you can use 6 LFF disks (larger capacities).

That’s a very impressive 2U server!

A DL165 G7 (1U) and SL165z G7 were also announced.  The DL165 G7 has 4 NIC’s and 24 DIMM slots for traditional and virtualization server computing.  Some of the other metrics are the same as above.  The SL165z G7 is for extreme scale out computing and constrained spaces.  It’s a cross between rack and blade servers.  It appears to be of a similar spec to the DL165 G7.

Some Useful Hyper-V Posts From the Last While

I’ve been going through my unread feeds from the last month or so (it’s been busy) and I’m posting links to the interesting ones here:

HP Ireland Event at Redstone: Data Protector in The Virtualised Environment

Redstone is one of Ireland’s leading enterprise hardware providers in Ireland – I’ll be open and admit that I’m a (happy) blade and storage customer.  They are running this event today in cooperation with HP Ireland.  The goodie bag will in no way influence me 🙂

Today’s event will focus on Data Protector, HP’s backup solution, and how it can be used in a virtualised environment.  The majority of the attendees are using EVA/VMware.  About 1/4 are using Hyper-V.  A couple are using Xen and a couple are using XP SAN.  No one here is using Lefthand.  About 1/5 are using Data Protector for their backups.

  • Virtualisation solves some problems but complicates backups.
  • We need to reduce backup costs – storage amounts.
  • We need to be able to reliably restore business critical data and secure sensitive data.

A common problem is that people rush head first into virtualisation without considering the strategy for backup.


  • VM level backup: The argument by the HP speaker is that this is resource intensive.
  • Host level backup: This “doesn’t” impact the performance of the host. Hmm.  There is an issue with recovered data consistency, e.g. is there Volume Shadow Copy integration to Windows VM’s?  SQL and Exchange don’t support this.

The speakers says Data Protector allows you to take both approaches to meet suitable requirements for each VM.

Data Protector 6.11 has VMware VCB and Hyper-V support.  The core product has a license.  It has the traditional bolt-on license approach.  Virtualisation requires an “Online Backup” license.  The Zero Downtime Backup allows integration into the snapshot features of your HP storage array.

Note: that’s probably the approach you’d go with for backup of a Hyper-V CSV due to the CSV coordinator/redirected I/O issue with host level backups – assuming this is supported by Data Protector.

For storage I/O intensive applications, Data Protector can take advantage of the ability to snapshot the targeted LUN’s.  You identify a LUN to backup, the SAN creates a copy, Data Protector backups up the copy while the primary continues to be used by the application/users.  This can be a partial copy for normal backup/recovery to save storage space/costs on the SAN.  You can do a full copy of the LUN for “instant recovery”, i.e. Data Protector restores file(s) from the copy of the LUN.  This requires additional per TB licensing.  The partial copy cannot do “instant recovery” because it links back to the original storage and isn’t completely independent.  There’s a cost for these two solutions so you save it for the mission critical, storage performance sensitive data/applications.  You can do this on a replicated partner SAN to do backups in your DR site instead of in the production site.  These solutions require the VSS integrations for the storage arrays.  Note that this isn’t for VM snapshots.

Zero Time Backup and Instant Recovery can be done in VMware if the VM uses raw device mapping (pass through disks).

Hyper-V Backup Methods

  • In VM agent
  • VSS system  provider snapshots
  • VSS hardware provider snapshots
  • Full restore of VM
  • Partial restore of files
  • Offline backups for VM’s
  • Zero downtime backup
  • Instant recovery

I would guess the last two require passthrough disks.  Might be a solution for SQL/Exchange VM’s.

Really, you will end up with a combination of backup methods across the data centre, depending on VM’s, applications, and backup/recovery times/impacts.

After coffee, we had some demos of VMware backups that didn’t go so well for the HP speaker.

In summary, Data Protector gives you some HP storage integrated backup options.  Be careful and ensure that servers, OS’s, and applications support the backup type being used.

Although HP and Microsoft have announced their “Forefront” virtualisation alliance, there’s still a lot of catch up going on with regards to Hyper-V knowledge and sharing.  Thanks to Redstone for organising this up in their scenic office in the Wicklow mountains – not exactly a bad place to be just after sunrise.

Technorati Tags: ,,,

Some HP and Hyper-V Links

Patrick Lownds, a fellow virtualisation MVP over in the UK, has provided a couple of useful links if you are running Hyper-V on HP equipment.  The first is a post on best practice guidance if you are running Hyper-V on a HP EVA SAN.  There is a whitepaper that goes through HP’s recommendations on this.  It was interesting to see they saw a fixed VHD’s get 7% more IOPS at 7% less latency than dynamic VHD’s.

The ProTips for HP are also available.  They’re not easy to find but Patrick provided me with a link.  The idea here is that HP SIM agents (which you should be installing, even if you don’t use the HP or other management software) detect hardware issues.  OpsMgr then picks up the alert and notifies VMM using the HP Pro Tips.  VMM can then take action, e.g. migrating VM’s from one host to another in the cluster.

Technorati Tags: ,,,