{"id":12876,"date":"2012-06-18T13:48:17","date_gmt":"2012-06-18T12:48:17","guid":{"rendered":"https:\/\/aidanfinn.com\/?p=12876"},"modified":"2012-06-18T13:48:17","modified_gmt":"2012-06-18T12:48:17","slug":"windows-server-2012-cluster-in-a-box-rdma-and-more","status":"publish","type":"post","link":"https:\/\/aidanfinn.com\/?p=12876","title":{"rendered":"Windows Server 2012 Cluster-In-A-Box, RDMA, And More"},"content":{"rendered":"<p>Notes taken from TechEd NA 2012 session <a href=\"http:\/\/channel9.msdn.com\/Events\/TechEd\/NorthAmerica\/2012\/WSV310\" target=\"_blank\">WSV310<\/a>:<\/p>\n<p><a href=\"http:\/\/channel9.msdn.com\/Events\/TechEd\/NorthAmerica\/2012\/WSV310\" target=\"_blank\"><img loading=\"lazy\" decoding=\"async\" style=\"background-image: none; padding-left: 0px; padding-right: 0px; display: block; float: none; margin-left: auto; margin-right: auto; padding-top: 0px; border: 0px;\" title=\"image\" src=\"https:\/\/aidanfinn.com\/wp-content\/uploads\/2012\/06\/image25.png\" border=\"0\" alt=\"image\" width=\"404\" height=\"182\" \/><\/a><\/p>\n<p><strong><span style=\"text-decoration: underline;\">Volume Platform for Availability<\/span><\/strong><\/p>\n<p>Huge amount of requests\/feedback from customers.\u00a0 MSFT spent a year focusing on customer research (US, Germany, and Japan) with many customers of different sizes.\u00a0 Came up with Continuous Availability with zero data loss transparent failover to succeed High Availability.<\/p>\n<p><strong><span style=\"text-decoration: underline;\">Targeted Scenarios<\/span><\/strong><\/p>\n<ul>\n<li>Business in a box Hyper-V appliance<\/li>\n<li>Branch in a box Hyper-V appliance<\/li>\n<li>Cloud\/Datacenter high performance storage server<\/li>\n<\/ul>\n<p><strong><span style=\"text-decoration: underline;\">What\u2019s Inside A Cluster In A Box?<\/span><\/strong><\/p>\n<p>It will be somewhat flexible.\u00a0 MSFT giving guidance on the essential components so expect variations.\u00a0 MSFT noticed people getting cluster networking wrong so this is hardwired in the box.\u00a0 Expansion for additional JBOD trays will be included.\u00a0 Office level power and acoustics will expand this solution into the SME\/retail\/etc.<\/p>\n<p><a href=\"https:\/\/aidanfinn.com\/wp-content\/uploads\/2012\/06\/image26.png\"><img loading=\"lazy\" decoding=\"async\" style=\"background-image: none; padding-left: 0px; padding-right: 0px; display: inline; padding-top: 0px; border: 0px;\" title=\"image\" src=\"https:\/\/aidanfinn.com\/wp-content\/uploads\/2012\/06\/image_thumb23.png\" border=\"0\" alt=\"image\" width=\"404\" height=\"226\" \/><\/a><\/p>\n<p>Lots of partners can be announced and some cannot yet:<\/p>\n<ul>\n<li>HP<\/li>\n<li>Fujitsu<\/li>\n<li>Intel<\/li>\n<li>LSI<\/li>\n<li>Xio<\/li>\n<li>And more<\/li>\n<\/ul>\n<p>More announcements to come in this \u201cwave\u201d.<\/p>\n<p><strong><span style=\"text-decoration: underline;\">Demo Equipment<\/span><\/strong><\/p>\n<p>They show some sample equipment from two Original Device Manufacturers (they design and sell into OEMs for rebranding).\u00a0 One with SSD and Infiniband is shown.\u00a0 A more modest one is shown too:<\/p>\n<p><a href=\"https:\/\/aidanfinn.com\/wp-content\/uploads\/2012\/06\/image27.png\"><img loading=\"lazy\" decoding=\"async\" style=\"background-image: none; padding-left: 0px; padding-right: 0px; display: block; float: none; margin-left: auto; margin-right: auto; padding-top: 0px; border: 0px;\" title=\"image\" src=\"https:\/\/aidanfinn.com\/wp-content\/uploads\/2012\/06\/image_thumb24.png\" border=\"0\" alt=\"image\" width=\"404\" height=\"236\" \/><\/a><\/p>\n<p>That bottom unit is a 3U cluster in a box with 2 servers and 24 SFF SAS drives.\u00a0 It appears to have additional PCI expansion slots in a compute blade.\u00a0 We see it in a demo later and it appears to have JBOD (mirrored Storage Spaces) and 3 cluster networks.<\/p>\n<p><strong><span style=\"text-decoration: underline;\">RDMA aka SMB Direct<\/span><\/strong><\/p>\n<p>Been around for quite a while but mostly restricted to the HPC space.\u00a0 WS2012 will bring it into wider usage in data centres.\u00a0 I wouldn\u2019t expect to see RDMA outside of the data centre too much in the coming year or two.<\/p>\n<p>RDMA enabled NICs also known as R-NICs.\u00a0 RDMA offloads SMB CPU processing in large bandwidth transfers to dedicated functions in the NIC.\u00a0 That minimises CPU utilisation for huge transfers.\u00a0 Reduces the \u201ccost per byte\u201d of data transfer through the networking stack in a server by bypassing most layers of software and communicating directly with the hardware.\u00a0 Requires R-NICs:<\/p>\n<ul>\n<li><a href=\"http:\/\/en.wikipedia.org\/wiki\/IWARP\" target=\"_blank\">iWARP<\/a>: TCP\/IP based.\u00a0 Works with any 10 GbE switch.\u00a0 RDMA traffic routable.\u00a0 Currently (WS2012 RC) limited to 10 Gbps per NIC port.<\/li>\n<li><a href=\"http:\/\/en.wikipedia.org\/wiki\/RDMA_over_Converged_Ethernet\" target=\"_blank\">RoCE<\/a> (RDMA over Converged Ethernet): Works with high-end 10\/40 GbE switches.\u00a0 Offers up to 40 Gbps per NIC port (WS2012 RC).\u00a0 RDMA not routable via existing IP infrastructure.\u00a0 Requires DCB switch with <a href=\"http:\/\/en.wikipedia.org\/wiki\/Ethernet_flow_control#Priority_flow_control\" target=\"_blank\">Priority Flow Control<\/a> (PFC).<\/li>\n<li><a href=\"http:\/\/en.wikipedia.org\/wiki\/InfiniBand\" target=\"_blank\">InfiniBand<\/a>:\u00a0Offers up to 54 Gbps per NIC port (WS2012 RC). Switches typically less expensive per port than 10 GbE.\u00a0 Switches offer 10\/40 GbE uplinks. Not Ethernet based.\u00a0 Not routable currently.\u00a0 Requires InfiniBand switches.\u00a0 Requires a subnet manager on the switch or on the host.<\/li>\n<\/ul>\n<p>RDMA can also be combined with SMB Multichannel for LBFO.<\/p>\n<p><a href=\"https:\/\/aidanfinn.com\/wp-content\/uploads\/2012\/06\/image28.png\"><img loading=\"lazy\" decoding=\"async\" style=\"background-image: none; padding-left: 0px; padding-right: 0px; display: block; float: none; margin-left: auto; margin-right: auto; padding-top: 0px; border: 0px;\" title=\"image\" src=\"https:\/\/aidanfinn.com\/wp-content\/uploads\/2012\/06\/image_thumb25.png\" border=\"0\" alt=\"image\" width=\"304\" height=\"248\" \/><\/a><\/p>\n<p>Applications (Hyper-V or SQL Server) do not need to change to use RDMA and make the decision to use SMB Direct at run time.<\/p>\n<p><strong><span style=\"text-decoration: underline;\">Partners &amp; RDMA NICs<\/span><\/strong><\/p>\n<ul>\n<li>Mellanox ConectX-3 Dual Port Adapter with VPI InfiniBand<\/li>\n<li>Intel 10 GbE iWARP Adapter For Server Clusters NE020<\/li>\n<li>Chelsio T3 line of 10 GbE Adapters (iWARP), have 2 and 4 port solutions<\/li>\n<\/ul>\n<p>We then see a live demo of 10 Giga<strong><span style=\"text-decoration: underline;\"><em>bytes<\/em><\/span><\/strong> (not Giga<strong><span style=\"text-decoration: underline;\"><em>bits<\/em><\/span><\/strong>) per second over Mellanox InfiniBand.\u00a0 They pull 1 of the 2 cables and throughput drops to 6,000 Gigabytes per second.\u00a0 Pop the cable back in and flow returns to normal.\u00a0 CPU utilisation stays below 5%.<\/p>\n<p><strong><span style=\"text-decoration: underline;\">Configurations and Building Blocks<\/span><\/strong><\/p>\n<ul>\n<li>Start with single Cluster in a Box, and scale up with more JBODs and maybe add RDMA to add throughput and reduce CPU utilisation.<\/li>\n<li>Scale horizontally by adding more storage clusters.\u00a0 Live Migrate workloads, spread workloads between clusters (e.g. fault tolerant VMs are physically isolated for top-bottom fault tolerance).<\/li>\n<li>DR is possible via Hyper-V Replica because it is storage independent.<\/li>\n<li>Cluster-in-a-box could also be the Hyper-V cluster.<\/li>\n<\/ul>\n<p>This is a flexible solution.\u00a0 Manufacturers will offer new refined and varied options.\u00a0 You might find a simple low cost SME solution and a more expensive high end solution for data centres.<\/p>\n<p><strong><span style=\"text-decoration: underline;\">Hyper-V Appliance<\/span><\/strong><\/p>\n<p>This is a cluster in a box that is both Scale-Out-File Server and Hyper-V cluster.\u00a0 The previous 2 node Quanta solution is set up this way.\u00a0 It\u2019s a value solution using Storage Spaces on the 24 SFF SAS drives.\u00a0 The space are mirrored for fault tolerance.\u00a0 This is DAS for the 2 servers in the chassis.<\/p>\n<p><strong><span style=\"text-decoration: underline;\">What Does All This Mean?<\/span><\/strong><\/p>\n<p>SAN is no longer your only choice, whether you are SME or in the data centre space.\u00a0 SMB Direct (RDMA) enables massive throughput.\u00a0 Cluster-in-a-Box enables Hyper-V appliances and Scale-Out File Servers in ready made kits, that are continuously available and scalable (up and out).<\/p>\n<div id=\"scid:0767317B-992E-4b12-91E0-4F059A8CECA8:a0f07b56-7600-4d8f-b863-edd7cfd804b8\" class=\"wlWriterEditableSmartContent\" style=\"margin: 0px; display: inline; float: none; padding: 0px;\">Technorati Tags: <a rel=\"tag\" href=\"http:\/\/technorati.com\/tags\/Event+Notes\">Event Notes<\/a>,<a rel=\"tag\" href=\"http:\/\/technorati.com\/tags\/Windows+Server+2012\">Windows Server 2012<\/a>,<a rel=\"tag\" href=\"http:\/\/technorati.com\/tags\/Storage\">Storage<\/a>,<a rel=\"tag\" href=\"http:\/\/technorati.com\/tags\/Failover+Clustering\">Failover Clustering<\/a>,<a rel=\"tag\" href=\"http:\/\/technorati.com\/tags\/Networking\">Networking<\/a><\/div>\n","protected":false},"excerpt":{"rendered":"<p>Notes taken from TechEd NA 2012 session WSV310: Volume Platform for Availability Huge amount of requests\/feedback from customers.\u00a0 MSFT spent a year focusing on customer research (US, Germany, and Japan) with many customers of different sizes.\u00a0 Came up with Continuous Availability with zero data loss transparent failover to succeed High Availability. Targeted Scenarios Business in &hellip; <a href=\"https:\/\/aidanfinn.com\/?p=12876\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Windows Server 2012 Cluster-In-A-Box, RDMA, And More&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_uf_show_specific_survey":0,"_uf_disable_surveys":false,"footnotes":""},"categories":[14],"tags":[176,63,80,99,118],"class_list":["post-12876","post","type-post","status-publish","format-standard","hentry","category-eventnotes","tag-eventnotes","tag-failover-clustering","tag-networking","tag-storage","tag-windows-server-2012"],"aioseo_notices":[],"jetpack_featured_media_url":"","amp_enabled":true,"_links":{"self":[{"href":"https:\/\/aidanfinn.com\/index.php?rest_route=\/wp\/v2\/posts\/12876","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/aidanfinn.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/aidanfinn.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/aidanfinn.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/aidanfinn.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=12876"}],"version-history":[{"count":0,"href":"https:\/\/aidanfinn.com\/index.php?rest_route=\/wp\/v2\/posts\/12876\/revisions"}],"wp:attachment":[{"href":"https:\/\/aidanfinn.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=12876"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/aidanfinn.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=12876"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/aidanfinn.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=12876"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}