I\u2019ve had a number of requests to specify the pieces of a solution where there is a Windows Server 2012 R2 Hyper-V cluster that uses SMB 3.0 to store virtual machines on a Scale-Out File Server with Storage Spaces (JBOD). So that\u2019s what I\u2019m going to try to do with this post. Note that I am not going to bother with pricing:<\/p>\n
Scale-Out File Server<\/span><\/strong><\/p>\n Remember that an SOFS is a cluster that runs a special clustered file server role for application data. A cluster requires shared storage. That shared storage will be one or more Mini-SAS-attached JBOD trays (on the Storage Spaces HCL list) with Storage Spaces supplying the physical disk aggregation and virtualization (normally done by SAN controller software).<\/p>\n On the blade versus rack server question: I always go rack server. I\u2019ve been burned by the limited flexibility and high costs of blades. Sure you can get 64 blades into a rack \u2026 but at what cost!?!?! FlexFabric-like solutions are expensive, and strictly speaking, not supported by Microsoft \u2013 not to mention they limit your bandwidth options hugely. The massive data centres that I\u2019ve seen and been in use 1U and 2U rack servers.\u00a0 I like 2U rack servers over 1U because 1U rack servers such as the R420 have only 1 full-height and 1 half-height PCI expansion slots. That half-height slot makes for tricky expansion.<\/p>\n For storage (and more) networking, I\u2019ve elected to go with RDMA networking. Here you have two good choices:<\/p>\n I\u2019ve ruled out RoCE. It\u2019s too damned complicated \u2013 just ask Didier Van Hoye (@workinghardinit<\/a>).<\/p>\n There will be two servers:<\/p>\n Regarding the JBODs:<\/p>\n Only use devices on the Microsoft HCL for your version of Windows Server. There are hardware features in these \u201cdumb\u201d JBODs that are required. And the testing process will probably lead to the manufacturer tweaking their hardware.<\/p>\n Not that although \u201cany\u201d dual channel SAS drive can be used, some firmwares are<\/em> actually better than others. DataOn Storage maintain their own HCL of tested HDDs & SSDs and HBAs. Stick with the list that your JBOD vendor recommends.<\/p>\n How many and what kind of drives do you need? That depends. My example is just that: an example.<\/p>\n How many trays do you need? Enough to hold your required number of drives \ud83d\ude00 Really though, if I know that I will scale out to fill 3 trays then I will buy those 3 trays up front. Why? Because 3 trays is the minimum required for tray fault tolerance with 2-way mirror virtual disks (LUNs). Simply going from 1 tray to 2 and then 3 won\u2019t do because data does not relocate.<\/p>\n Also remember that if you want tiered storage then there is a minimum number of SSDs<\/a> (STRONGLY) recommended per tray.<\/p>\n Regarding using SATA drives: DON\u2019T DO IT! The available interposer solution is strongly discouraged, even by DataOn.\u00a0 If you really need SSD for tiered storage then you really need to pay (through the nose).<\/p>\n Here\u2019s my EXAMPLE<\/span><\/em><\/strong> configuration:<\/p>\n And that\u2019s the SOFS, servers + JBODs with disks.<\/p>\n Just to remind you: it\u2019s a sample spec. You might have one JBOD, you might have 4, or you might go with the 60 disk slot models. It all depends.<\/p>\n Hyper-V Hosts<\/span><\/strong><\/p>\n My hosting environment will consist of one Hyper-V cluster with 8 nodes. This could be:<\/p>\n One of the benefits of SMB 3.0 storage is that a shared folder is more flexible than a CSV on a SAN LUN. There are more sharing options, and this means that Live Migration can span the traditional boundary of storage without involving Shared-Nothing Live Migration.<\/p>\n Regarding host processors, the L2\/L3 cache plays a huge role in performance. Try to get as new a processor as possible. And remember, it\u2019s all Intel or all AMD; do not mix the brands.<\/p>\n There are lots of possible networking designs for these hosts. I\u2019m going to use the design<\/a> that I\u2019ve implemented in the lab at work, and it\u2019s also one that Microsoft recommends. A pair or rNICs (iWARP) will be used for the storage and cluster networking, residing on the same two VLANs as the cluster\/storage networks that the SOFS nodes are on. Then two other NICs are going to be used for host and VM networking. These two NICs could be 1 GbE or 10 GbE or faster, depending on the needs of your VMs. I\u2019ve got 4 pNICs to play with so I will team them.<\/p>\n <\/p>\n There are no HBAs in the Hyper-V hosts; the storage (SOFS) is accessed via SMB 3.0 over the rNICs.<\/p>\n Other Stuff<\/span><\/strong><\/p>\n Hmm, we\u2019re going to need:<\/p>\n And there\u2019s probably other bits. For example, you might run a 2-node cluster for System Center and other management VMs. The nodes would have 32-64 GB RAM each. Those VMs could be stored on the SOFS or even on a JBOD that is directly attached to the 2 nodes with Storage Spaces enabled. You might run a server with lots of disk as your backup server. You might opt to run a pair of 1U servers are physical domain controllers for your infrastructure.<\/p>\n I recently priced up a kit, similar to above. It came in much cheaper than the equivalent blade\/SAN configuration, which was a nice surprise. Even better was that the SOFS had 3 times more storage included than the SAN in that pricing!<\/p>\n I\u2019ve had a number of requests to specify the pieces of a solution where there is a Windows Server 2012 R2 Hyper-V cluster that uses SMB 3.0 to store virtual machines on a Scale-Out File Server with Storage Spaces (JBOD). So that\u2019s what I\u2019m going to try to do with this post. Note that I … Continue reading “A Kit\/Parts List For A WS2012 R2 Hyper-V Cluster With DataOn SMB 3.0 Storage”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_uf_show_specific_survey":0,"_uf_disable_surveys":false,"footnotes":""},"categories":[20,1],"tags":[180,181,91,99,100,195,120],"class_list":["post-15886","post","type-post","status-publish","format-standard","hentry","category-hyper-v","category-uncategorized","tag-hardware","tag-hyper-v","tag-scale-out-file-server","tag-storage","tag-storage-spaces","tag-virtualisation","tag-windows-server-2012-r2"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n
\n
\n
\n
\n
\n
\n