I\u2019ve illustrated a high-level implementation below. Mirrored Storage Spaces are being used as the back-end storage. Two LUNs are created on this storage. A cluster is built from FS1 and FS2, and connected to the shared storage and the 2 LUNs. Each LUN is added to the cluster and converted to Cluster Shared Volume (CSV). Thanks to a new feature in WS2012 R2, CSV ownership (the CSV coordinator automatically created and managed role) is automatically load balanced across FS1 and FS2. Let\u2019s assume, for simplicity, that CSV1 is owned by FS1 and CSV2 is owned by FS2.<\/p>\n
The File Server for Application Data role (SOFS) is added to the cluster and named as SOFS1. A share is added to CSV1 called CSV1-Share, and a share called CSV2-Share is added to CSV2.<\/p>\n
Any number of Hyper-V hosts\/clusters can be permitted to use both or either share. For simplicity, I have illustrated just Host1.<\/p>\n Name Resolution<\/u><\/strong><\/p>\n Host1 wants to start up a VM called VM1. The metadata of VM1 says that it is stored on \\SOFS1CSV1-Share. Host1 will do a DNS lookup for SOFS1 when it performs an initial connection. This query will return back all of the IP addresses of the nodes FS1 and FS2.<\/p>\n Tip: Make sure that the storage\/cluster networks of the SOFS nodes are enabled for client connectivity in Failover Cluster Manager. You\u2019ll know that this is done because the NICs\u2019 IP addresses will be registered in DNS with additional A records for the SOFS CAP\/name.<\/em><\/p>\n Typically in this scenario, Host1 will have been given 4-6 addresses for the SOFS role. It will perform a kind of client based round robin, randomly picking one of the IP addresses for the initial connection. If that fails, another one will be picked. This process continues until a connection is made or the process times out.<\/p>\n Now SMB 3.0 kicks in. The SMB client (host) and the SMB server (SOFS node) will negotiate capabilities such as SMB Multichannel and SMB Direct.<\/p>\n Tip: Configure SMB Multichannel Constraints to control which networks will be used for storage connectivity.<\/em><\/p>\n Initial Connection<\/u><\/strong><\/p>\n There are two scenarios now. Host1 wants to use CSV1-Share so the best possible path is to connect to FS1, the owner of the CSV that the share is stored on. However, the random name resolution process could connect Host1 to FS2.<\/p>\n Let\u2019s assume that Host1 connects to FS2. They negotiate SMB Direct and SMB Mulitchannel and Host1 connects to the storage of the VM and starts to work. The data flow will be as illustrated below.<\/p>\n Mirrored Storage Spaces offer the best performance. Parity Storage Spaces should not be used<\/em><\/u><\/strong> for Hyper-V. Repeat: Parity Storage Spaces SHOULD NOT BE USED<\/font><\/em><\/u><\/strong> for Hyper-V. However, Mirrored Storage Spaces in a cluster, such as a SOFS, are in permanent redirected IO mode.<\/p>\n What does this mean? Host1 has connected to SOFS1 to access CSV1-Share via FS2. CSV1-Share is on CSV1. CSV1 is owned by FS1. This means that Host1 will connect to FS2, and FS2 will redirect the IO destined to CSV1 (where the share lives) via FS1 (the owner of CSV1).<\/p>\n Don\u2019t worry; this is just the initial connection to the share. This redirected IO will be dealt with in the next step. And it won\u2019t happen again to Host1 for this share once the next step is done.<\/p>\n Note: if Host1 had randomly connected to FS1 then we would have direct IO and nothing more would need to be done.<\/p>\n You can see why the cluster networks between the SOFS nodes needs to be at least as fast as the storage networks that connect the hosts to the SOFS nodes. In reality, we\u2019re probably using the same networks, converged to perform both roles, making the most of the investment in 10 GbE, or faster and possibly RDMA.<\/p>\n SMB Client Redirection<\/u><\/strong><\/p>\n There is another WS2012 R2 feature that works along-side CSV balancing. The SMB server, running on each SOFS node, will redirect SMB client (Host1) connections to the owner of the CSV being accessed. This is only done if the SMB client has connected to a non-owner of a CSV.<\/p>\n After a few moments, the SMB server on FS2 will instruct Host1 that for all traffic to CSV1, Host1 should connect to FS1. Host1 seamlessly redirects and now the traffic will be direct, ending the redirected IO mode.<\/p>\n TIP: Have 1 CSV per<\/em> node in the SOFS.<\/p>\n What About CSV2-Share?<\/u><\/strong><\/p>\n What if Host1 wants to start up VM2 stored on \\SOFS1CSV2-Share? This share is stored on CSV2 and that CSV is owned by Host1. Host1 will again connect to the SOFS for this share, and will be redirected to FS2 for all traffic related to that share. Now Host1 is talking to FS1 for CSV1-Share and to FS2 for CSV2-Share.<\/p>\n TIP: Balance the placement of VMs across your CSVs in the SOFS. VMM should be doing this for you anyway if you use it. This will roughly balance connectivity across your SOFS nodes.<\/em><\/p>\n And that is how SOFS, SMB 3.0, CSV balancing, and SMB redirection give you the best performance with clustered mirrored Storage Spaces.<\/p>\n<\/p>\n I thought I\u2019d write a post on how traffic connects and flows in a Windows Server 2012 R2 implementation of Hyper-V with the storage being Hyper-V over SMB 3.0 on a WS2012 R2 Scale-Out File Server (SOFS). There are a number of pieces involved. Understanding what is going on will help you in your design, … Continue reading “Flow Of Storage Traffic In Hyper-V Over SMB 3.0 to WS2012 R2 SOFS”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_uf_show_specific_survey":0,"_uf_disable_surveys":false,"footnotes":""},"categories":[20],"tags":[63,181,80,99,195,120],"class_list":["post-15785","post","type-post","status-publish","format-standard","hentry","category-hyper-v","tag-failover-clustering","tag-hyper-v","tag-networking","tag-storage","tag-virtualisation","tag-windows-server-2012-r2"],"aioseo_notices":[],"jetpack_featured_media_url":"","amp_enabled":true,"_links":{"self":[{"href":"https:\/\/aidanfinn.com\/index.php?rest_route=\/wp\/v2\/posts\/15785","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/aidanfinn.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/aidanfinn.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/aidanfinn.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/aidanfinn.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=15785"}],"version-history":[{"count":0,"href":"https:\/\/aidanfinn.com\/index.php?rest_route=\/wp\/v2\/posts\/15785\/revisions"}],"wp:attachment":[{"href":"https:\/\/aidanfinn.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=15785"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/aidanfinn.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=15785"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/aidanfinn.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=15785"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}![\"image\"](\"https:\/\/aidanfinn.com\/wp-content\/uploads\/2013\/12\/image2.png\" "\\"image\\"")
<\/a> <\/p>\n<\/p>\n<\/a><\/p>\n<\/a> <\/p>\n