{"id":11945,"date":"2011-12-20T13:11:00","date_gmt":"2011-12-20T13:11:00","guid":{"rendered":"https:\/\/aidanfinn.com\/?p=11945"},"modified":"2011-12-20T13:11:00","modified_gmt":"2011-12-20T13:11:00","slug":"great-article-on-hyper-v-and-numa","status":"publish","type":"post","link":"https:\/\/aidanfinn.com\/?p=11945","title":{"rendered":"Great Article on Hyper-V and NUMA"},"content":{"rendered":"

Any software designer\/engineer needs to be aware of how Non-Uniform Memory Access (NUMA) impacts the performance of services that will run on that hardware.  This goes double for virtualisation administrators, and here\u2019s why.<\/p>\n

NUMA is a hardware design feature that divides CPUs and memory in a physical server into NUMA nodes.  You get the best performance when a process uses memory and CPU from within the same NUMA node.  When a process requires more memory, but the current NUMA node is full, then it\u2019ll get memory from another NUMA node and that comes at a performance cost to that process, and possibly all other processes on that physical server.<\/p>\n

And that\u2019s why virtualisation engineers need to be aware of this.  In Hyper-V we have Dynamic Memory.  In VMware, there are other techs that do similar (but work differently) things to add memory to a VM under the covers.  When there\u2019s contention in a NUMA node, a VM will be given additional memory from a different NUMA node \u2013 and then performance will drop.<\/p>\n

When present this topic, NUMA causes a lot of confusion.  Microsoft gave us a rather badly out-dated formula for calculating NUMA node sizes.  It\u2019s actually a hardware specification (that all OS\u2019s and hypervisors have to deal with) so the only really accurate way to determine NUMA node layouts is via PerfMon or chatting to the hardware vendor.  In the meantime, I stumbled across this fantastic article<\/a> by Benjamin Athawes.  Benjamin explains NUMA superbly and talks about how to determine what\u2019s in your hardware.<\/p>\n

In the Hyper-V world, we can disable NUMA node spanning in the host settings.  That\u2019s thanks to how Dynamic Memory works \u2013 there isn\u2019t an over-commitment that must be lived up to by the hypervisor.  If we see lots of spanning that impacts performance, then we have choices:<\/p>\n