{"id":18059,"date":"2015-05-06T18:32:27","date_gmt":"2015-05-06T18:32:27","guid":{"rendered":"https:\/\/aidanfinn.com\/?p=18059"},"modified":"2015-05-06T19:44:20","modified_gmt":"2015-05-06T19:44:20","slug":"ignite-2015hyper-v-storage-performance-with-storage-quality-of-service","status":"publish","type":"post","link":"https:\/\/aidanfinn.com\/?p=18059","title":{"rendered":"Ignite 2015&ndash;Hyper-V Storage Performance with Storage Quality of Service"},"content":{"rendered":"<p><em>I am live blogging this session so hit refresh to see more.<\/em><\/p>\n<p>Speakers: Senthil Rajaram and Jose Barreto.<\/p>\n<p>This session is based on what\u2019s in TPv2. There is a year of development and FEEDBACK left, so things can change. If you don\u2019t like something \u2026 tell <em>Microsoft<\/em>.<\/p>\n<h2>Storage Performance<\/h2>\n<ol>\n<li>You need to measure to shape<\/li>\n<li>Storage control allows shaping<\/li>\n<li>Monitoring allows you to see the results \u2013 do you need to make changes?<\/li>\n<\/ol>\n<h2>Rules<\/h2>\n<ul>\n<li>Maximum Allowed: Easy \u2013 apply a cap.<\/li>\n<li>Minimum Guaranteed: Not easy. It\u2019s a comparative value to other flows. How do you do fair sharing? A centralized policy controller avoids the need for complex distributed solutions. <\/li>\n<\/ul>\n<h2>The Features in WS2012 R2<\/h2>\n<p>There are two views of performance:<\/p>\n<ul>\n<li>From the VM: what the customer sees \u2013 using perfmon in the guest OS<\/li>\n<li>From the host: What the admin sees \u2013 using the Hyper-V metrics<\/li>\n<\/ul>\n<p>VM Metrics allow performance data to move with a VM. (get-vm \u2013name VM01)&#160; | Measure-VM).HardDiskMetrics \u2026. it\u2019s Hyper-V Resource Metering \u2013 Enable-VMResourceMetering.<\/p>\n<h2>Normalized IOPS<\/h2>\n<ul>\n<li>Counted in 8K blocks \u2013 everything is a multiple of 8K.<\/li>\n<li>Smaller than 8K counts as 1<\/li>\n<li>More than 8K counted in multiples, e.g 9K = 2.<\/li>\n<\/ul>\n<p>This is just an accounting trick. Microsoft is not splitting\/aggregating IOs.<\/p>\n<p>Used by:<\/p>\n<ul>\n<li>Hyper-V Storage Performance Counters<\/li>\n<li>Hyper-V VM Metrics (HardDiskMetrics)<\/li>\n<li>Hyper-V Storage QoS<\/li>\n<\/ul>\n<h2>Storage QoS in WS2012 R2<\/h2>\n<p>Features:<\/p>\n<ul>\n<li>Metrics \u2013 per VM and VHD<\/li>\n<li>Maximum IOPS per VHD<\/li>\n<li>Minimum IOPS per VHD \u2013 alerts only<\/li>\n<\/ul>\n<p>Benefits:<\/p>\n<ul>\n<li>Mitigate impact of noisy neighbours<\/li>\n<li>Alerts when minimum IOPS are not achieved<\/li>\n<\/ul>\n<p>Long and complicated process to diagnose storage performance issues.<\/p>\n<h2>Windows Server 2016 QoS Instroduction.<\/h2>\n<p>Moving from managing IOPS on the host\/VM to managing IOPS on the storage system.<\/p>\n<p>Simple storage QoS system that is installed in the base bits. You should be able to observe performance for the entire set of VMs. Metrics are automatically collected, and you can use them even if you ar enot using QoS. No need to log into every node using the storage subsystem to see performance metrics. Can create policies per VM, VHD, service or tenant. You can use PoSH or VMM to manage it.<\/p>\n<p>This is a SOFS solution. One of the SOFS nodes is elected as the policy manager \u2013 a HA role. All of the nodes in the cluster share performance data, and the PM is the \u201cthinker\u201d.<\/p>\n<ol>\n<li>Measure current capacity at the compute layer.<\/li>\n<li>Measure current capacity at the storage layer<\/li>\n<li>use algorithm to meet policies at the policy manager<\/li>\n<li>Adjust limits and enforce them at the compute layer<\/li>\n<\/ol>\n<p>In TP2, this cycle is done every 4 seconds. Why? Storage and workloads are constantly changing. Disks are added and removed. Caching makes \u201ctotal IOPS\u201d impossible to calculate. The workloads change \u2026 a SQL DB gets a new index, or someone starts a backup. Continuous adjustment is required.<\/p>\n<h2>Monitoring<\/h2>\n<p>On by default You can query the PM to get a summary of what\u2019s going on right now.<\/p>\n<p>Available data returned by a PoSH object:<\/p>\n<ul>\n<li>VHD path<\/li>\n<li>VM Name<\/li>\n<li>VM Host name<\/li>\n<li>VM IPOS<\/li>\n<li>VM latency<\/li>\n<li>Storage node name<\/li>\n<li>Storage node IOPS<\/li>\n<li>Storage node latency<\/li>\n<\/ul>\n<p>Get-StorageQoSFlow \u2013 performance of all VMs using this file server\/SOFS<\/p>\n<p>Get0StorageQoSVolume \u2013 performance of each volume on this file server\/SOFS<\/p>\n<p>There are initiator (the VM\u2019s perspective) metric and storage metrics. Things like caching can cause differences in initiator and storage metrics.<\/p>\n<p>Get-StorageQoSFlow | Sort InitiatorIPOS | FT InitiarorName, InitiatorIIOPS, InitiatorLatency<\/p>\n<p>Working not with peaks\/troughs but with averages over 5 minutes. The Storage QoS metrics, averaged over the last 5 minutes, are rarely going to match the live metrics in perfmon.<\/p>\n<p>You can use this data: export to CSV, open in Excel pivot tables<\/p>\n<h2>Deploying Policies<\/h2>\n<p>Three elements in a policy:<\/p>\n<ul>\n<li>Max: hard cap<\/li>\n<li>Min: Guaranteed allocation if required<\/li>\n<li>Type: Single or Multi-instance<\/li>\n<\/ul>\n<p>You create policies in one place and deploy the policies.<\/p>\n<p>Single instance: An allocation of IOPS that are shared by a group of VMs. Multi-instance: a performance tier. Every VM get\u2019s the same allocation, e.g. max IOPS=100 and each VM gets that.<\/p>\n<h2>Storage QoS works with Shared VHDX<\/h2>\n<p>Active\/Active: Allocation split based on load. Active\/Passive: Single VM can use full allocation.<\/p>\n<p>This solution works with Live Migration.<\/p>\n<h2>Deployment with VMM<\/h2>\n<p>You can create and apply policies in VMM 2016. Creaate in Fabric &gt; Storage &gt; QoS Policies. Deploy in VM Properties &gt; Hardware Configuration &gt; &lt;disk&gt; &gt; Advanced. You can deploy via a template.<\/p>\n<p>PowerShell<\/p>\n<p>New-StorageQoSPolicy \u2013CimSession FS1 \u2013Name sdjfdjsf \u2013PolicyType MultiInstance \u2013 MaximumIOPS 200<\/p>\n<p>Get-VM \u2013Name VM01 | Get-VMHardDiskDrive | Set-VMHardDiskDrive \u2013QosPolicy $Policy<\/p>\n<p>Get-StorageQoSPolicy \u2013Name sdfsdfds | Get-StorageQoSFlow \u2026 see data on those flows affected by this policy. Pulls data from the PM.<\/p>\n<h2>Demo<\/h2>\n<p>The way they enforce max IOPS is to inject latency in that VM\u2019s storage. This reduces IOPS.<\/p>\n<h2>Designing Policies<\/h2>\n<ul>\n<li>No policy: no shaping. You\u2019re just going to observe uncontrolled performance. Each VM gets at least 1 IOPS<\/li>\n<li>Minimum Only: A machine will get at least 200 IOPS, IF it needs it. VM can burst. Not for hosters!!! Don\u2019t set false expectations of maximum performance.<\/li>\n<li>Maximum only: Price banding by hosters or limiting a noisy neighbour.<\/li>\n<li>Minimum &lt; Maximum, e.g. between 100-200: Minimum SLA and limited max.<\/li>\n<li>Min = Max: VM has a set level of performance, as in Azure.<\/li>\n<\/ul>\n<p>Note that VMs do not use min IOPS if they don\u2019t have the workload for it. It\u2019s a min SLA.<\/p>\n<h2>Storage Health Monitoring<\/h2>\n<p>If total Min of all disks\/VMs exceeds the storage system then:<\/p>\n<ul>\n<li>QoS does it\u2019s best to do fair share based on proportion.<\/li>\n<li>Raises an alert.<\/li>\n<\/ul>\n<p>In WS2016 there is 1 place to get alerts for SOFS called Storage health Monitoring. It\u2019s a new service on the SOFS cluster. You\u2019ll get alerts on JBOD fans, disk issues, QoS, etc. The alerts are only there while the issue is there, i.e. if the problem goes away then the alert goes away. There is no history.<\/p>\n<p>Get-StorageSubSystem *clsuter* | Debug-StorageSubSystem.<\/p>\n<p>You can register triggers to automate certain actions.<\/p>\n<p>Right now \u2013 we spend 10x more than we need to to ensure VM performance. Storage QoS reduces spend by using a needle to fix issues instead of a sledge hammer. We can use intelligence to solve performance issues instead of a bank account.<\/p>\n<p>In Hyper-V converged solution, the PM and rate limiters live on the same tier. Apparently there will be support for a SAN \u2013 I\u2019m unclear on this design.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I am live blogging this session so hit refresh to see more. Speakers: Senthil Rajaram and Jose Barreto. This session is based on what\u2019s in TPv2. There is a year of development and FEEDBACK left, so things can change. If you don\u2019t like something \u2026 tell Microsoft. Storage Performance You need to measure to shape &hellip; <a href=\"https:\/\/aidanfinn.com\/?p=18059\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Ignite 2015&ndash;Hyper-V Storage Performance with Storage Quality of Service&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_uf_show_specific_survey":0,"_uf_disable_surveys":false,"footnotes":""},"categories":[14],"tags":[181,80,99,195,137],"class_list":["post-18059","post","type-post","status-publish","format-standard","hentry","category-eventnotes","tag-hyper-v","tag-networking","tag-storage","tag-virtualisation","tag-windows-server-2016"],"aioseo_notices":[],"jetpack_featured_media_url":"","amp_enabled":true,"_links":{"self":[{"href":"https:\/\/aidanfinn.com\/index.php?rest_route=\/wp\/v2\/posts\/18059","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/aidanfinn.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/aidanfinn.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/aidanfinn.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/aidanfinn.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=18059"}],"version-history":[{"count":18,"href":"https:\/\/aidanfinn.com\/index.php?rest_route=\/wp\/v2\/posts\/18059\/revisions"}],"predecessor-version":[{"id":18077,"href":"https:\/\/aidanfinn.com\/index.php?rest_route=\/wp\/v2\/posts\/18059\/revisions\/18077"}],"wp:attachment":[{"href":"https:\/\/aidanfinn.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=18059"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/aidanfinn.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=18059"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/aidanfinn.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=18059"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}