Storage | Aidan Finn, IT Pro

TechEdNA 2013 – Application Availability Strategies for the Private Cloud

Speakers: Jose Barreto, Steven Ekren

Pre-session question … How far can a cluster stretch? You can have your heartbeat up to 1 minute timeout. They recommend no more than 10-20 seconds. However there is a license mobility limit – it is pretty long distance, but it does exist.

Moving Physical to the Private Cloud (Virtual)

Many ways to P2V from rebuilt, disk2vhd, backup/restore, VMM, and on and on and on.

VMs can be HA on Hyper-V. Cost reductions and mobility by virtualization. Easier backup. Easier deployment. Easier monitoring. Flexibility. Self-service. Measurability. Per-VM/VHD VM replication is built in with Hyper-V Replica. And on and on and on.

VM Monitoring added in WS2012 Failover Clustering

2 levels of escalated action in response to a failure trigger:

Guest level HA recovery
Host level HA recovery

Off by default and requires configuration. Watch for an alert, say from a service. If service fails, cluster gets the alert and restarts the service. If within an hour, the cluster gets the same alert again, it’ll fail it over (shut down) to another host.

Requires that the VM is WS2008 R2 or later and in the same domain as the hosting Hyper-V cluster.

In the private cloud:

Guest OS admin configures the failure triggers
Recovery from host is configured by the cloud admin

The process works through the Hyper-V heartbeat integration component in the guest OS. An “application critical flag” goes back to the parent partition via VMMS, and escalated in the host via the VM resource in the cluster, to the Cluster Service.

You can enable VM Monitoring in WS2012 in the VM properties (cluster) in Settings. The cluster will still get a signal, if configured in the guest OS, but it is ignored. Basically cloud admin can disable the feature, and it ignores what the tenant does in their VM.

Event ID 1250 will be registered in System log with FailoverClustering source when the application critical flag is sent.

We can set up a trigger for a service failure or an event.

Add-ClusterVMMonitoredItem … Get-, Remove-, Reset- are run by a guest OS admin in the VM.

You can also hit Configure Monitoring action on a VM in Failover Cluster Manager on the cloud. Assumes you have admin rights in the VM.

Guest Clustering

We can create guest OS clusters. Protects against faults in the guest layer, e.g. BSOD, registry issue, etc. Also allows preventative maintenance with high SLAs.

Can use: iSCSI, virtual fiber channel, or SMB 3.0 shared storage.

Guest Clustering and VM Monitoring

You can use both together.

Set cluster service restart action to none for 2nd and 3rd failure in the guest cluster node OS. First failure is left at Restart the Service.

Then from the host site, enable VM monitoring for the guests’ Cluster Service.

Demo of virtual SOFS

Steven kills the cluster service on a SOFS node using Process Explorer. The service restarts. Video being streamed from the SOFS via that node pauses and resumes maybe 2-3 seconds later. He kills the service a second time. The host cluster shuts down the VM and fails it over.

Thorough Resource Health Check Interval defaults to 1 minute in the VM properties in Failover Cluster Manager. You can reduce this if you need to, maybe 20 seconds. Don’t make it too often, because the check does run a piece of code and that would be very inefficient.

Jose comes on stage.

Shared Virtual Disks

Before WS2012 R2, the only way we could do guest clustering was by surfacing physical/cloud storage to the tenant layer, or by deploying virtual file servers/iSCSI. First is insecure and inflexible, second is messy. Hosting companies just won’t want to do it – and most will refuse.

With WS2012 R2, VMs can share a VHDX file as their shared data disk(s). It is a shared SAS device from the VM’s perspective. It is for data disks only.

There are 2 scenarios supported:

Using CSV to store the VHDX
Using SMB to store the VHDX

The storage location of the CSV must be available to all hosts that guest cluster nodes will be running on.

This solution isolates the guests/tenants from your hosts/cloud fabric.

Deploying Shared VHDX

Use:

Hyper-V Manager
PowerSHell
VMM 2012 R2

Think about:

Anti-affinity, availability sets in VMM service templates. Keep the guests on different hosts so you don’t have a single point of failure.
Watch out for heartbeats being too low.

Deploy the data disk on the SCSI controller of the VMs. Enable sharing in the Advanced features of the VHDX in the VM settings.

In the VM, you just see a shared SAS disk. You can use an older version of Windows … 2012 and 2012 R2 will be supported. This is limited by time to test older versions.

PowerShell:

New-VHD
Add-VMHardDiskDrive …. –ShareVirtualDisk < repeat this on all the guest cluster VMs
Get-VMHardDiskDrive … | ft VMName, Path, ControllerType, SupportPersistentReservations < the latter setting indicates that it is shared if set to True.

In VMM service template tier properties, you can check Share The Disk Across The Service Tier in the VHDX properties.

Inside the VM, it just looks like a typical disk in Disk Management, just like in physical cluster.

Tip: use different VHDX files for your different data volumes in the guest OS cluster. It gives you more control and flexibility. Stop being lazy and do this!

The hosts must be 2012. The guests are 2012 and 2012 R2, with the latest integration components.

This is only VHDX – it uses the metadata feature of the disk to store persistent reservation information. Can use fixed or dynamic, but not differencing.

Backup

Guest-based backup only. Host based-backups and snapshots of the shared VHDX are not supported. Same restrictions as with guest clusters using physical storage.

Storage Migration of Shared VHDX

This is not supported – it is being referenced by multiple VMs. You can Live Storage Migrate the other VM files, but just not the shared data VHDX of the guest cluster.

You can Live Migrate the VMs.

Comparing Guest Cluster Options

Troubleshooting

Performance counters: Added new counters to PerfMon
Event Viewer: Hyper-V-Shared-VHDX
Filter Manager (FLTMC.EXE): The Shared VHDX filter can be looked at – svhdxflt
Actual binaries of the filer: svhdxflt.sys and pvhdparsersys

Online Resize

You can hot resize a non-shared VHDX in WS2012 R2. You cannot hot resize a shared VHDX.

You can hot-add a shared VHDX.

Unsupported bonus scenario

Technorati Tags: Event Notes,Windows Server 2012 R2,Hyper-V,Storage,Networking,Failover Clustering

TechEd NA 2013–Storage Spaces Performance

Speaker: Brian Matthew

Start was some metrics achieved stuff. Summary: Lots of IOPS.

Hardware

It’s simple and cost effective. Goes from basic to OLTP workloads.

Capabilities Overview

Storage Pools support ReFS

2 * JBODs. We create a single storage pool to aggregate all 48 disks ( 2 * 24 in this example. We create 1 * 2-way mirror spaces and 1 * parity space.

Flexible resilient storage spaces.
Native data striping maximizes performance
Enclosure awareness with certified hardware.
Data Integrity Scanner (aka “scrubber”) with NTFS and ReFS
Continuous Availability with Windows Server failover clustering – SOFS

Data is spread around the disks in the storage pool. They parallelize the rebuild process.

8 * 3 TB disk test bed. Test the failure of the disk. Can rebuild in 50 minutes, with > 800 MB/s rebuild throughput. The line is that hot spare is no longer necessary in WS2012 R2. Hmm. Must look into that.

Scale-Out Example

Note: CSV scales out linearly

Match workload characteristics to drives

Capacity optimized drives have lower performance. Higher TB/$
High performance drives has lower capacity/host. Higher IOPS/$

Can we seamlessly merge these?

Tiered Storage Spaces

A single virtual disk can use the best of both types of disk. High capacity for colder slices of data. High speed for hotter slices of data.

The most compelling ratio appears to be 4 to 12 SSDs in a 60 slot device, with the rest of the disks being HDDs.

In the background, the file system actively measures the activity of file slices. Transparently moves hot slices to the SSD tier, and cold slices to the HDD tier.

Tiering (analysis and movement) is done daily. The schedule is configurable (change time, do it more than daily). The slices are 1 MB in size. So tracking watches 1 MB slices, and tiering is done on 1 MB slices.

Administrators can pin entire files to specified tiers. Example, move a VDI parent VHDX to the SSD tier.

Write-Back Cache

Per virtual disk, persistent write cache. It smoothens out write bursts to a virtual disk. Uses the SSD capacity of the pool for increased IOPS capacity. Configurable using PowerShell. Great for Hyper-V which needs write-through, instead of battery powered write cache.

PowerShell Demo

Get-PhysicalDisk to list the possible disks to use “CanPool”attribute.

$disks = Get-PhyscalDisks

New-StoragePool …. $disks

Get-StoragePool to see the disks. Look at FriendlyName and MediaType attributes.

$SSD_Tier = New-StorageTier … list the SSDs

$HDD_Tier = New-StorageTier … list the HDDs

$vd1 = New-VirtualDisk …. –StorageTiers @($ssd_tier, $hdd_tier) –StorageTierSizes @(150GB, 1.7TB) ….

Now we have a drive with automated scheduled storage tiering.

Pins some files using Set-FileStorgeTier

Optimize-Volume –DriveLetter E –TierOptimize ….. this will force the maintenance task to run and move slices.

Demo: Write-Back Cache

He increases the write workload to the disk. A quick spike and then the SSD takes over. Then increases again and again, and the write-back cache absorbs the spikes.

Question: How many tiers are supported in WS2012 R2? 2. But the architecture will allow MSFT to increase this in later releases if required.

Right now, certified clustered storage spaces from:

DataOn
RAID Incorporated
Fujitsu

Takeaways

WS2012 R2 is a key component in the cloud:cost efficient
Scalable data access: capacity and performance
Continuously available
Manageable from Server Manager, PoSH, and SCVMM (including SOFS bare metal deployment from template.

Q&A

No docs on sizing Write-Back Cache. They want the WBC to be not too large. Up to 10 GB is being recommended right now. You can reconfigure the size of the WBC after the fact … so monitor it and change as required.

On 15K disks: Expensive and small. Makes sense to consider SSD + 7.5K disks in a storage pool rather than SSD + 15 K in a storage pool.

He can’t say it, but tier 1 manufacturers are scared *hitless of Storage Spaces. I also hear one of them is telling porky pies to people on the Expo floor re the optimization phase of Storage Spaces, e.g. saying it is manual.

Is there support for hot spares? Yes, in WS2012 and R2. Now MSFT saying you should use space capacity in the pool with parallelized repair across all disks in the pool, rather than having a single repair point.

DeFrag is still important for contiguous data access.

If I have a file on the SSD tier, and the tier is full, writes will continue OK on the lower tier. The ReFS integrity stream mechanism can find best placement for a block. This is integrated with tiered storage spaces.

On adding physical disks to the storage space: old data is not moved: instant availability. New writes are sent to the new disks.

A feature called dirty region table protects the storage space against power loss caused corruption.

Should hard drive caches be turned off? For performance: turn it off. For resilience, turn it on. Note, a cluster will bypass the disk cache with write-through.

There is some level of failure prediction. There are PoSH modules for detecting issues, e.g. higher than normal block failure rates, or disks that are slower than similar neighbours.

Ah, the usual question: Can the disks in a storage space span data centers. The members of a storage pool must be connected to all nodes in a SOFS via SAS, which makes that impossible. Instead, have 2 different host/storage blocks in 2 sites, and use Hyper-V Replica to replicate VMs.

Virtual Disk Deployment Recommendations

When to use Mirror, Parity, or Simple virtual disks in a storage space?

A storage space will automatically repair itself when a drive fails – and then it becomes resilient again. That’s quick thanks to parallelized repair.

Personal Comment

Love hearing a person talk who clearly knows their stuff and is very clear in their presentation.

—

Holy crap, I have over a mile to walk to get to the next storage session! I have to get out before the Q&A ends.

Technorati Tags: Windows Server 2012 R2,Storage

TechEd 2013: System Center 2012 R2–Virtual Machine Manager

Speaker: Vijay Tewari, VMM PM.

Boostrapping a repeatable architecture

VMM becomes the heart of the data centre. You deploy everything from VMM console/library. For example, MSFT will be supplying service templates for deploying the reset of System Center from VMM.

Network Architecture

A sample one:

Storage

Using SOFS service templates, SMB 3.0 management, SMI-S (including fiber channel support), VMM 2012 R2 can manage the entire storage stack from bare metal to zoning/permissioning.

Demo

Host Profiles has become Physical Computer Profiles. You can create a file server profile for a SOFS bare metal deployment. He reaches out to the BMC (DRAC, ILO, etc) to discover, power up, and deploy the OS of the cluster nodes. If the process completed, a new SOFS would be running and managed by VMM. Now you can use VMM to provision and permission file shares. Once done, you can start to place/move VMs on the file share on the permitted hosts.

Note: you don’t touch the file servers, log into them, use Server Manager, use a PoSH cmdlet. It’s all done from the VMM console. Very sweet.

See Datacenter Abstraction Layer (DAL).

Synthetic Fiber Channel In The Guest

VMM 2012 R2 adds support for guest fiber channel in Hyper-V. Uses SMI-S to talk to the SAN. Involves 2 things:

Project a fiber channel virtual adapter in the guest
You need to be able to program the fiber channel network

Simplified zone management from the VMM console.

Storage

Offloaded data transfer is now supported in VMM 2012 R2 to provision resources from the library.
VMM supports WS2012 R2 Hyper-V to create guest clusters using a shared VHDX. Remember the VHDX is stored on shared storage (CSV or SMB). MSFT uses this kind of SQL cluster for testing SysCtr. It’s a check box: Share this disk across the service tier … yes, you can deploy a guest cluster from a service template.

New in Service Templates: the first node online will initialize the cluster, and additional nodes join the cluster. Service templates understand the need for different tasks on the first and subsequent nodes. In the demo, he talks about how SQL can be installed on the guest cluster as part of the service template.

IP Address Management

You can create networks in VMM and IPAM will detect it. Or you can use IPAM to model your networks and VMM will pull in the configuration.

Top of Rack Switches

More DAL. This is where VMM can configure/manage physical switches using OMI. In the demo, a host cannot respond to a ping. In VMM, the host is non-compliant. The reasoning is that the required VLAN is not piped through the switch port to the host NIC. There is a “remediate” button – press it and VMM can reach out to an OMI switch to fix the issue …. assuming you have a RunAs account for the switch. Otherwise you beat your network admin with some Cat5 cables until he relents.

Hybrid Networking

This builds on things like virtual switch extensions, NVGRE, etc. The ability to move a VM from one network to another without changing the IP, and the VM stays online using HNV.

Windows Azure Pack is shown in the demo. Vijay spins up a network in a hosting company public cloud. He configures the IP stack of the new virtual subnet (a subset of a VM network). A site-site tunnel (VPN) is configured. Remember, WS2012 R2 RRAS will do this for us (NGVRE gateway too).

He configures IBGP for routing, and then configures the VPN connection properties (IP, shared key, etc). Now he has extended his on premise network into the hosting company.

Gateway Service Templates

An out of the box SCVMM 2012 R2 service template will automate the deployment of the WS2012 R2 NVGRE gateway.

Hyper-V Recovery Manager

This is Hyper-V Replica management via a new SaaS product in the Azure cloud (Recovery Services). It is in preview at the moment. A provider (an agent) is installed in the VMM servers in production and DR sites – VMM must manage the production cloud and the DR cloud, with a VMM server in each site. This only does management; all data replication goes directly from production to DR site, never going to Azure.

He configures cloud to cloud replication policies. Now from in the VMM console, he can enable replication on a per-VM basis using Enable Recovery or Disable Recovery in the ribbon. Replica VMs have a slightly different icon than production VMs.

HRM can be used to create recovery plans and be used to invoke them.

Operations Manager Dashboard Monitoring

A new OpsMgr MP, with rich dashboards. Demo: Drive down into the fabric health. Clicks on a network node and looks at the network vicinity dashboard to browse the health of the network. Can diagnose networking issues in the VMM console.

Summary

Built on features of WS2012 and added support for WS2012 R2 features.

Technorati Tags: Event Notes,System Center,VMM,Storage,Networking,Cloud,Hyper-V,Windows Server 2012 R2

Windows Server 2012 R2 Hyper-V Feature List Glossary

I’m going to do my best (no guarantees – I only have one body and pair of ears/eyes and NDA stuff is hard to track!) to update this page with a listing of each new WS2012 R2 Hyper-V and Hyper-V Server 2012 R2 (and related) feature as it is revealed by Microsoft (starting with TechEd North America 2013). Note, that the features of WS2012 can be found here.

This list was last updated on 05/September/2013.

3^rd party Software Defined Networking	Is supported by the extensibility of the virtual switch.
Automatic Guest Activation	Customers running WS2012 R2 Datacenter can automatically activate their WS2012 R2 guests without using KMS. Works with OEM and volume licenses. Great for multi-tenant clouds.
Azure Compatibility	Azure is running the same Hyper-V as on-premise deployments, giving you VM mobility from private cloud, to hosted cloud, to Microsoft Azure.
Built-In NVGRE Gateway	A multi-tenant aware NVGRE gateway role is available in WS2012 R2. Offers site-site VPN, NAT for Internet access, and VM Network to physical network gateway.
Clustering: Configurable GUM Mode	Global Update Manager (GUM) is responsible for synchronizing cluster resource updates. With Hyper-V enabled, all nodes must receive and process an update before it is committed to avoid inconsistencies.
Clustering: Larger CSV Cache Percentage	WS2012 allows a maximum of 20% RAM to be allocated to CSV Cache. This is 80% in WS2012 R2.
Clustering: CSV Load Balancing	CSV ownership (coordinators) will be automatically load balanced across nodes in the cluster.
Clustering: CSV & ReFS	ReFS is supported on CSV. Probably still not preferable over NTFS for most deployments, but it is CHKDSK free!
Clustering: Dynamic Witness	The votes of cluster nodes are automatically changed as required by the cluster configuration. Enabled by default. This can be used to break 50/50 votes when a witness fails.
Clustering: Hyper-V Cluster Heartbeat	Clusters running Hyper-V have a longer heartbeat to avoid needless VM failovers on latent/contended networks. SameSubnetThreshold is 10 (normally 5) and CrossSubnetThreshold is 20 (normally 5).
Clustering: Improved logging	Much more information is recorded during host add/remove operations.
Clustering: Pause action	Pausing a node no longer will use Quick Migration for “low” priority VMs by default; Live Migration is used as expected by most people. You can raise the threshold to force Quick Migration if you want to.
Clustering: Proactive Server Service Health Detection	The health of a destination host will be verified before moving a VM to another host.
Clustering: Protected Networks	Virtual NICs are marked as being on protected networks by default. If a virtual NICs’ virtual switch becomes disconnected then the cluster will Live Migrate that VM to another host with a healthy identical virtual switch.
Clustering: Virtual Machine Drain on Host Shutdown	Shutting down a host will cause all virtual machines to Live Migrate to other hosts in the cluster.
Compressed Live Migration	Using only idle CPU resources on the host, Hyper-V can compress Live Migration to make it quicker. Could provide up to 2x migrations on 1 GbE networks.
Cross-Version Live Migration	You can perform a Live Migration from WS2012 to WS2012 R2. This is one-way, and enables zero-downtime upgrades from a WS2012 host/cluster to a WS2012 R2 host/cluster.
Dynamic Mode NIC Teaming	In addition to Hyper-V Port Mode and Address Hashing. Uses “flowlets” to give fine-grained inbound and outbound traffic.
Enhanced Session Mode	The old Connect limited KVM access to a VM. Now Connect can use Remote Desktop that is routed via the Hyper-V stack, even without network connection to the VM. Copy/paste and USB redirection are supported. Disabled on servers and enabled by Client Hyper-V by default.
Generation 2 VM	A G2 virtual machine is a VM with no legacy “hardware”. It uses UEFI boot, has no emulated devices, boots from SCSI, and can PXE boot from synthetic NIC. You cannot convert from G1 VM (UEFI I am guessing).
HNV Diagnostics	A new PoSH cmdlet enables an operator to diagnose VM connectivity in a VM Network without network access to that VM.
HNV: Dynamic Learning of CAs	Hyper-V Network Virtualization can learn the IPs of VM Network VMs. Enables guest DHCP and guest clustering in the VM Network.
HNV: NIC Teaming	Inbound and outbound traffic can traverse more than one team member in a NIC team for link aggregation.
HNV: NVGRE Task Offloads	A new type of physical NIC will offload NVGRE de- and encapsulation from the host processor.
HNV: Virtual Switch extensions	The HNV filter has been included in the Hyper-V Virtual Switch. This enables 3^rd party extensions to work with HNV CAs and PAs.
Hyper-V Replica Extended Replication	You can configure a VM in Site A to replicate to Site B, and then replicate it from Site B to Site C.
Hyper-V Replica Finer Grained Interval controls	You can change the replication interval from the default 5 minutes to every 30 seconds or every 15 minutes.
IPAM	IP Address Management was extended in WS2012 R2 to do management of physical and virtual networking with built-in integration into SCVMM 2012 R2.
Linux Dynamic Memory	All features of Dynamic Memory are supported on WS2012 R2 hosts with the up to date Linux Integration Services.
Linux Kdump/kexec	Allows you to create kernel dumps of Linux VMs.
Linux Live VM backup	You can backup a running Linux VM with no pause, with file system “freeze”, giving file system consistency. Linux does not have VSS.
Linux Specification of Memory Mapped I/O (MMIO) gap	Provides fine grained control over available RAM for virtual appliance manufacturers.
Linux Non-Maskable Interrupt (NMI)	Allows delivery of manually triggered interrupts to Linux virtual machines running on Hyper-V.
Linux Video Driver	A Synthetic Frame Buffer driver for Linux guest OSs will provide improved performance and mouse support.
Live Resizing of VHDX	You can expand or shrink (if there’s un-partitioned space) a VHDX attached to a running VM. It must be SCSI attached. This applies to Windows and Linux.
Live Virtual Machine Cloning	You can clone a running virtual machine. Useful for testing and diagnostics.
Remote Live Monitoring	Remote monitoring of VM network traffic made easier with Message Analyzer.
Service Provider Foundation (SPF)	The SPF is used to provide an API in-front of SCVMM. It is required for the Windows Azure Pack. A hosting company can share their infrastructure with clients, who can interact with SPF via on-premise System Center – App Controller.
Shared VHDX	Up to 8 VMs can share a VHDX (on shared storage like CSV/SMB) to create guest clusters. Appears like a shared SAS drive.
SMB Live Migration	This feature uses SMB to perform Live Migration over 10 GbE or faster networks. It uses SMB Multichannel if there are multiple Live Migration networks. SMB Direct is used if RDMA is available. SMB Multichannel gives the fastest VM movement possible, and SMB Direct offloads the work from the CPU. Now moving that 1 TB RAM VM doesn’t seem so scary!
SMB 3.0: Automatic rebalancing of Scale-Out File Server clients	SMB clients of the scalable and continuously available active/active SOFS are rebalanced across nodes after the initial connection. Tracking is done per-share for better alignment of server/CSV ownership.
SMB 3.0: Bandwidth controls	QoS just sees SMB 3.0. New filters for default, live migration, and virtual machine allow you to manage bandwidth over converged networks.
SMB 3.0: Improved RDMA performance	Improves performance for small I/O workloads such as OLTP running in a VM. Very noticeable on 40/56 Gbps networks.
SMB 3.0: Multiple SMB instances on SOFS	The Scale-Out File Server has an additional SMB instance for CSV management, improving scalability and overall reliability. Default instance handles SMB clients.
Storage Spaces: Tiered Storage	You can mix 1 tier of SSD with 1 tier of HDD to get a blend of expensive extreme speed and economic capacity. You define how much (if any) SSD and how much HDD a virtual disk will take from the pool. Data is promoted/demoted in the tiers at 1am by default. You can pin entire files to a tier.
Storage Spaces: Parallelized Restore	Instead of using slow host spare disks in a pool, you can use the cumulative write IOPS of the pool to restore virtual disk fault tolerance over the remaining healthy disks. The replacement disk is seen as new blank capacity.
Storage Spaces: Write-Back Cache	Hyper-V is write-through, avoiding controller caches on writes. With tiered storage, you get Write-Back Cache. The SSD tier can absorb spikes in write activity. Supported with CSV.
Storage QoS	You can set an IOPS limit on individual virtual hard disks to avoid one disk consuming all resources, or to price-band your tenants. Minimum alerts will notify you if virtual hard disks cannot get enough storage bandwidth.
System Center alignment	System Center and Windows Server were developed together and will be released very closely together.
Network Diagnostics	New PowerShell tools for testing the networking of VMs, including Get-VMNetworkAdapter, Test-NetConnection, Test-VMNetworkAdapter,a nd Ping -P.
VDI & Deduplication	WS2012 R2 can be enabled in VDI scenarios (only) where the VMs are stored on dedicated (only) WS2012 R2 storage servers.
Virtual Machine Exports	You can export a VM with snapshots/checkpoints
Virtual Switch Extended Port ACLs	ACLs now include the socket port number. You can now configure stateful rules that are unidirectional and provide a timeout parameter. Compatibility with Hyper-V Network Virtualization.
vRSS	Virtual Receive Side Scaling leverages DVMQ on the host NIC to enable a VM to use more than 1 vCPU to process traffic. Improves network scalability of a VM.
Windows Azure Pack	This was previously called Windows Azure Services for Windows Server, and is sometimes called “Katal”. This is based on the source code of the Azure IaaS portal, and allows companies (such as hosting companies) to provide a self-service portal (with additional cloud traits) for their cloud.

Technorati Tags: Windows Server 2012 R2,Hyper-V,Virtualisation,Networking,Storage

Event Notes – What’s New In Windows Server 2012 R2?

Speaker Jeff Woolsey

The Cloud OS Vision

The Private Cloud is Windows Server & System Center. Virtualisation is not cloud. P2V didn’t’ change management. Look at the traits of a cloud in the NIST definition. Cloud-centric management layers change virtualisation into a cloud. That’s what SysCtr 2012 and later do to virtualization layers: create clouds.

Microsoft’s public cloud is Azure, powered by Hyper-V, a huge stress (performance and scalability) on a hypervisor.

Hosting companies can also use Windows Azure Pack on Windows Server & System Center to create a cloud. That closes the loop … creating 1 consistent platform across public and private, on premise, in Microsoft, and in hosting partners. The customer can run their workload everywhere.

Performance

The absolute best way to deploy MSFT biz apps is on Hyper-V: test, support, validation, optimization, test, test, test. They test everything on Hyper-V and Azure, every single day. 25,000 VMs are created every day to do automated unit tests of Windows Server.

In stress tests, Exchange (beyond recommended scale) tested well within Exchange requirements on Hyper-V. Over 1,000,000 IOPS from a Hyper-V VM in a stress test.

Storage

If you own a SAN, running WS2012 or newer is a no brainer: TRIM, UNMAP, ODX.

Datacenter without Boundaries

Goal number 1.

They wanted integrated high performance virtualization platform. Reduce complexity, cost, and downtime. Ease deployment. Flexible.

Automatic VM activation. Live VM export/cloning. Remote Access via VMBus. Online VHDX resize. Live Migration compression. Live Migration over RDMA. More robust Linux support.

Ben Armstrong on demo patrol:

Storage QoS. You can cap the storage IOPS of a VM, on a per hard disk basis.

Linux has full dynamic memory support on WS2012 R2. Now we can do file system consistent backup of Linux VMs without pausing them. Don’t confuse it with VSS – Linux does not have VSS. It’s done using a file system freeze.

You can do shared VHDX to create 100% virtual production ready guest clusters. The shared VHDX appears as a SAS connected disk in the guest OSs. Great for cloud service providers to enable 100% self service. Store the VHDX on shared storage, e.g. CSV or SMB 3.0 to support Live Migration … best practice is that the guest cluster nodes be on different hosts Smile

End of Ben in this session.

Demystifying Storage Spaces and SOFS

I‘ll recommend you watch the session. Jeff uses a storage appliance to explain a file server with Storage Spaces. He’ll probably do the same with classic SAN and scale-out file server.

Matt McSpirit comes up.

He’s using VMM to deploy a new file server cluster. He’s not using Failover Clustering or Server Manager. He can provision bare metal cluster members. Like the process of deploying bare metal hosts. The shares can be provisioned and managed through VMM, as in 2012 SP1. You can add new bare-metal hosts. There is a configurable thin provisioning alert in the GUI – OpsMgr with the MP for VMM will alert on this too.

Back to Jeff.

Changes of Guest Clustering

It’s a problem for service providers because you have previously needed to provide a LUN to the customer. Hoster’s just can’t do it because of customisation. Hoster can’t pierce the hosting boundary, and customer is unhappy. With shared VHDX, the shared storage resides outside the hoster boundary is the tenant domain. It’s completely virtualised and perfect for self-service.

SDN

The real question should be: Why deploy software defined networking (Hyper-V Network Virtualization). The primary answer is “you’re a hosting company that wants multi-tenancy with abstracted networking for seamless network convergence for hybrid clouds”. Should be a rare deployment in the private cloud – unless you’re friggin huge or in the acquisition business.

WS2012 R2 will feature a built-in multi-tenant NVGRE (Hyper-V Network Virtualisation or Software Defined Newtorking) gateway. Now you don’t need F5’s vapourware or the Iron Networks appliance to route between VM Networks and physical networks. You choose the gateway when creating your VM Network (create VM Network Wizard, Connectivity). VPN, BGP and NAT are supported.

You can deploy the gateway using a VMM Service Template.

You can use OMI based rack switches, eg. Arista, to allow VMM to configure your Top Of Rack (TOR) switches.

Hyper-V Replica

HVR broadens your replication … maybe you keep your synchronous replication for some stuff if you made the investment. But you can use HVR for everything else – hardware agnostic (both ends). Customers love it. Service providers should offer it as a service. But service providers also want to replicate.

Hyper-V Recovery Manager gives you automation and orchestration of VMM-managed HVR. You install a provider in the VMM servers in site A and site B. Then enable replication in VMM console. Replication goes direct from site A to B. Hyper-V Recovery Manager gives you the tools to create, implement, and monitor the failover plans.

You can now choose your replica interval which defaults to every 5 minutes. Alternatives as 30 seconds and 15 minutes.

Scenario 1: customer replicates from primary hosts (a) to hosts (b) across the campus. Lots of pipe in the campus so do 30 seconds replica intervals. Then replicates from primary DR (b) site to secondary and remote DR site (c). Lots of latency and bandwidth issues, so go for every 15 minutes.

Scenario 2: SME replicates to hosting company every 5 minutes. Then the hosting company replicates to another location that is far away.

Michael Leworthy comes up to demo HRM. We get a demo of the new HVR wizards. Then HRM is shown. HRM workflows allow you to add manual tasks, e.g. turn on the generator.

Technorati Tags: Event Notes,Windows Server 2012 R2,Linux,Hyper-V,Virtualisation,Backup,Storage,DR

TechEd NA 2013: Keynote – The Post VMware Era

I am love blogging this session so please hit refresh to get the latest notes.

Pre-show, everything is running nice and smoothly. I got in at 7am and check-in was running nicely (lots of desks) but I was even luckier by being able to register at the media desk. One breakfast later and we were let into the keynote hall after just a few minutes’ wait, and I went into the press reserved area up to the left of the front. We had lots of handlers there … handy when my ultrabook refused to see the TechEd network and I had to find other means to connect.

Rock music was playing, and then came out a classic New Orleans brass band to liven things up. All we needed was some beer Smile

Lots of well known media types start appearing in the press section as the band plays “The Saints Come Marching In” (at least until the 49ers D crushes them).

TechEd 2014 is in Houston next year. Hopefully there is a route that does not include Dallas Fort Worth airport.

Brad Anderson

A pre-video where “the bits have been stolen” and Brad goes all James Bond to get them back, chasing the baddies in an Aston Martin while wearing a tux. The Windows USB key is being unsuccessfully uploaded (BitLocker to go)? And he recovers his shades Smile And he drives out onto the stage with the Aston Martin. Best keynote entrance ever.

All new versions of datacenter products:

-Devices
-Services to light up devices and enable users (BYOD)
-Azure and Visual Stuid to create great apps
-SQL Server to unlock insights into data
-The cloud platform: what enables the entire stack

Iain McDonald (Windows Core)
Makes the kernel, virtualisation, ID, security, and file system for all the products using Windows Core (Azure, Windows 8, Phone, XBoxOne, etc). Windows is our core business, he says. In other words, Windows lets you get your stuff. Windows 8 is out for 8 months and sold 100,000,000 copies in that time.

A Windows 8 blurb video, and during that a table full of Windows 8 devices comes out. Confirms that Windows 8.1 will be compatible, out this year, and free. Preview bits out on June 26th. Personalized background on the Start Screen. Some biz features will be shown:

Start Screen control: We can lock down tile customization. You can set up specific apps and setup. Set up a template machine. It’s an XML file export-startlayout. Set a GPO: Start Screen Layout. Paste a UNC path to the XML file. GPO refresh on the user machine, and the start screen is locked out. Windows 8.1 Industry line (embedded) does a lot of lock down and customization stuff for hard appliances.
Mirrorcast: a powerpoint display technology. He pairs a machine with a streamless wiring device. Now he presents from a tablet. I want this now. I need this now. Much better than VGA over Wifi – which just flat out doesn’t work with animated systems like Windows 8 Start Screen.
Wifi Printer with NFC. Tab the tablet and it pairs with the printer, and adds the device/printer. The demo gods are unkind Eventually he goes into Mail and can open an attachment side-by-side (50/50 split). And he sends the attachment to a printer. This is why wifi in big demo rooms does not work: the air is flooded – the print doesn’t appear as expected.
Surface Pro is up next. Can build VPN into apps in 8.1. Can work with virtual smart card for multi-factor authentication.

On the security front:

Moving from a defensive posture to an offensive posture in the security space.
8” Atom powered Acer tablet (see below).
Toshiba super hi-res Kira ultrabook

Back to Brad

1.2 billion consumer devices sold since last TechEd. 50% of companies told to support them. 20-somethings think BYOD is a right not a privilege. IT budgets are not expanding to support these changes.

Identity: Windows Server AD syncs with and blends with Windows Azure Active Directory (WAAD). Windows Intune connects to on-premise ConfigMgr (System Center). Manage your devices where they live, with a single user ID. Don’t try to manage BYOD or mobile devices using on-premise systems – that just flat-out doesn’t work.

Aston Martin has lots of widely distributed and small branch offices (retail). Windows Intune is perfect to manage this, and they use it for BYOD.

Windows Server and System Center 2012 R2 are announced, as is a new release of Windows Intune (wave E). Get used to the name of Windows Server and System Center. Microsoft has designed for the cloud, and brought it on-premises. Scalability, flexibility, and dependability.

Out comes Molly Brown, Principal Development Lead.

Workplace Join: She is going to show some new solutions in 2012 R2. Users can work on the devices they want while you remain in control She has a Windows 8.1 tablet and logs into a TemShare site. Her access is deined. She can “join her workplace”. This is like joining a domain. Policy is applied to her identity rather than to the device. Think of this as a modern domain join – Anderson. She joins the workplace in Settings -Network – Workplace. She enters her corporate email address and password, and then she has to prove herself, via multifactor authentication, e.g. a phone call. All she has to do is press the # key when prompted. Now she can view the Sharepoint site.

To get IT apps, she can enrol her device for management via Workplace (into Intune). Now she can (if the demo works – wifi) access IT published apps through Intune.

Work Folders: A new feature of WS2012 R2. Users have access to all their files across all their devices. Files replicated to file servers in the datacenter and out to all devices owned by the user. Relies on the device being enrolled.

You can easily leave the workplace and turn off management with 2 taps. All your personal stuff is left untouched. BYOD is made much easier.

Remote wipe is selective, only removing corporate assets from personal devices.

App and device management is Intune. You brand your service to the business, and manage cross-platform devices including Apple and Android (I found IOS device management to actually the be easier than Windows!).

So you empower end users, unify the environment, and secure the business.

Back to Brad

Apps. Devs want rapid lifecycles and flexibility. Need support for cross-platform deployment. And data, any size. And make it secure while being highly available.

On to the public cloud and Azure sales pitch. A dude from Easyjet comes out. I hope everyone has paid to use the priority lane to exit the hall. He talks about cloud scalability.

Scott Guthrie

Corp VP for Windows Azure. Cloud great for dev/test because of agility without waiting on someone to do something for you. Same hypervisor on premise in Hyper-V as in Azure, so you can choose where your app is deployed (hybrid cloud).

No charge for stopped VMs in Windows Azure from now on. You can stop it and start it, knowing that you’ve saved money by shutting it down. Now there is pro-rated per-minute billing. Great for elastic workload. You can use MSDN licenses on Azure for no charge. Or you can deploy pre-created images in the portal. A new rate for MSDN subscribers to run any number of VMs in Azure at up to 97% discount. MSDN subscribers get monthly credits ($50 pro, $100 premium, $150 ultimate), and you can use these VMs for free for dev/test purposes. The portal has been updated today to see what your remaining credit balance is. I might finally fire up an Azure VM.

http://aka.ms/azurecontest .. MSDN competition for subscribers that deploy an Azure app. Could win an Aston Martin.

Brian Harry

Technical Fellow – Appliance lifecycle management

Next version of Visual Studio and TFS 2013 later this year. Preview on June 26th in line with Build. How to help devs to get from idea-implementation-into customer hands-feedback and all over again. New cloud load test service from the cloud. Create the test in VS/TFS, upload it to the cloud, and it runs from there.

SQL Server 2014 is announced. Hybrid scenarios for Azure. Lots of memory work – transaction processing in RAM. Edgenet is an early adopter. They need reliable stock tracking, without human verification. This feature has moved away from once/day stock inventory batch jobs to realtime.

PixelSense monster touch TV comes out. And they start doing touch-driven analytics on the attendees. A cool 3D map of the globe allows them to visualize attendees based on regions.

Back to Brad

Windows Server 2012 R2 and System Center 2012 R2 out at the end of the year, and the previews out in June. These are based on the learnings from Azure for you to use on-premise or to build your own public cloud. Same Hyper-V as in Azure. This gives us consistency across clouds – ID, data, services across all clouds with no conversion.

Windows Azure Pack for Windows Server. This layers on top of System Center and System Center. This is the new name for Katal by the looks of it. Same portal as Azure. Get density and Service Bus on top of WSSC 2012 R2. Users deploy services on the cloud of choice.

Clare Henry, Director of Product Management comes out. You get a stack to build your clouds. Demo: and we see the Katal portal, renamed to Windows Azure Pack. Creates a VM from a gallery as a self-service user. Can deploy different versions of a VM template. All the usual number/scalability and network configuration options.

The self-service empowers the end user, builds on top of WSSC for automation, and allows the admin hands-off total control.

On to the fabric and the infrastructure. Here’s the cool stuff.

Jeff Woolsey

WSSC 2012 R2 is about agility. Storage Spaces. Automated storage tiering is coming to Storage Spaces using SSD and HDD. Bye bye EMC. That gave 16x performance improvement from 7K to 124K IOPS.

Deduplication. Enabling Dedup will actually improve the performance of VDI. We now have a special VDI mode for Hyper-V VDI. It is NOT FOR SERVER VMs. Dedup will actually 2x the performance of those VDI VMs.

Live Migration just got unreal. WS2012 R2 Live Migration can use resources of the host to do compression (for 10 GbE or less). It’ll use some resources if available … it won’t compress if there’s resource contention – to prioritise VMs.

Now LM can use SMB Direct over RDMA. And SMB Multichannel. You get even faster LMs over 10 GbE or faster networks using RDMA.

Hyper-V Replica now supports: Site A – Site B – Site C replication, e.g. replicate to local DR, and from local DR to remote DR.

I wonder how VMware’s Eric Gray will try to tap dance and spin that faster Live Migration isn’t needed. They don’t have anything close to this.

Hyper-V Recovery Manager gives you orchestration via the cloud. DR was never this easy.

Brad is back

Blue led a new development cadence. What they’ve accomplished in 9 months is simply amazing.

We can reduce the cost of infrastructure again, increase flexibility, and be heroes.

Post Event Press Conference

Hybrid cloud was the core design principal from day 1 – Brad Anderson. Organizations should demand consistency – it gives flexibility to move workloads anywhere. It’s not just virtualization – storage, Identity, networks, the whole stack.

Scott Guthrie: private cloud will probably continue forever. But don’t make forks in the road that limit your flexibility.

Windows Azure Pack is confirmed as the renamed next generation version of Katal. A new feature is the ability to use Service Bus on Windows Server, with a common management portal for private and public. No preview release date.

Thanks to Didier Van Hoye for this one. Stockholders not too confident in VMware this morning. Is it a coincidence that Microsoft stole their lunch money this morning?

To quote Thomas Maurer: we are entering the post-VMware era.

What is in Windows 8.1 for the enterprise? It is the "next vision of Windows 8". "No compromises to corporate IT".

Making your PC a hotspot is a new feature. BYOD is huge in the 8.1 release, enabled by Windows Intune. The Workplace join and selective resets are great. And the file sync feature controlled by the biz is also a nice one. XP End of Life: what is the guidance… the official line will be “the easiest path to Windows 8.1 is Windows 8”. Actually they are being realistic about Windows 7 deployment being the norm. Mobility and touch scenarios should be future proofed with the right devices. Windows 8 is the natural OS choice for this.

On System Center, it is now WSSC, Windows Server and System Center as a combined solution, designed to work at data center scale. It’s one holistic set of capabilities. Watch for networking and storage being lit up at scale via System Center. The new version of Orchestrator is entire based on PowerShell.

Technorati Tags: Event Notes,Windows Server 2012 R2,System Center,Storage,Virtualisation,Cloud,Networking

MVP Carsten Rachfahl Interviews Microsoft Storage Guru Jose Barreto

Fellow Hyper-V MVP, Carsten Rachfahl (@hypervserver) published an interview he did with Microsoft Program Manager Jose Barreto. In the video, they talk about the massive changes that SMB 3.0 brings to storage of Hyper-V virtual machines, and other application data.

Technorati Tags: Windows Server 2012,Storage,Hyper-V,Virtualisation,Networking

A Converged Networks Design For Hyper-V On SMB 3.0 Storage With RDMA (SMB Direct)

When you are done reading this post, then see the update that I added for SMB Live Migration on Windows Server 2012 R2 Hyper-V.

Unless you’ve been hiding under a rock for the last 18 months, you might know that Windows Server 2012 (WS2012) Hyper-V (and IIS and SQL Server) supports storing content (such as virtual machines) on SMB 3.0 (WS2012) file servers (and scale-out file server active/active clusters). The performance of this stuff goes from matching/slightly beating iSCSI on 1 GbE, to crushing fiber channel on 10 GbE or faster.

Big pieces of this design are SMB Multichannel (think simple, configuration free & dynamic MPIO for SMB traffic) and SMB Direct (RDMA – low latency and CPU impact with non-TCP SMB 3.0 traffic). How does one network this design? RDMA is the driving force in the design. I’ve talked to a lot of people about this topic over the last year. They normally over think the design, looking for solutions to problems that don’t exist. In my core market, I don’t expect lots of RDMA and Infiniband NICs to appear. But I thought I’d post how I might do a network design. iWarp was in my head for this because I’m hoping I can pitch the idea for my lab at the office. Smile

On the left we have 1 or more Hyper-V hosts. There are up to 64 nodes in a cluster, and potentially lots of clusters connecting to a single SOFS – not necessarily 64 nodes in each!

On the right, we have between 2 and 8 file servers that make up a Scale-Out File Server (SOFS) cluster with SAS attached (SAN or JBOD/Storage Spaces) or Fiber Channel storage. More NICs would be required for iSCSI storage for the SOFS, probably using physical NICs with MPIO.

There are 3 networks in the design:

The Server/VM networks. They might be flat, but in this kind of design I’d expect to see some VLANs. Hyper-V Network Virtualization might be used for the VM Networks.
Storage Network 1. This is one isolated and non-routed subnet, primarily for storage traffic. It will also be used for Live Migration and Cluster traffic. It’s 10 GbE or faster and it’s already isolated so it makes sense to me to use it.
Storage Network 2. This is a second isolated and non-routed subnet. It serves the same function as Storage Network 2.

Why 2 storage networks, ideally on 2 different switches? Two reasons:

SMB Multichannel: It requires each multichannel NIC to be on a different subnet when connecting to a clustered file server, which includes the SOFS role.
Reliable cluster communications: I have 2 networks for my cluster communications traffic, servicing my cluster design need for a reliable heartbeat.

The NICs used for the SMB/cluster traffic are NOT teamed. Teaming does not work with RDMA. Each physical rNIC has it’s own IP address for the relevant (isolated and non-routed) storage subnet. These NICs do not go through the virtual switch so the easy per-vNIC QoS approach I’ve mostly talked about is not applicable. Note that RDMA is not TCP. This means that when an SMB connection streams data, the OS packet scheduler cannot see it. That rules out OS Packet Scheduler QoS rules. Instead, you will need rNICs that support Datacenter Bridging (DCB) and your switches must also support DCB. You basically create QoS rules on a per-protocol-basis and push them down to the NICs to allow the hardware (which sees all traffic) to apply QoS and SLAs. This also has a side effect of less CPU utilization.

Note: SMB traffic is restricted to the rNICs by using the constraint option.

In the host(s), the management traffic does not go through the rNICs – they are isolated and non-routed. Instead, the Management OS traffic (monitoring, configuration, remote desktop, domain membership, etc) all goes through the virtual switch using a virtual NIC. Virtual NIC QoS rules are applied by the virtual switch.

In the SOFS cluster nodes, management traffic will go through a traditional (WS2012) NIC team. You probably should apply per-protocol QoS rules on the management OS NIC for things like remote management, RDP, monitoring, etc. OS Packet Scheduler rules will do because you’re not using RDMA on these NICs and this is the cheapest option. Using DCB rules here can be done but it requires end-to-end (NIC, switch, switch, etc, NIC) DCB support to work.

What about backup traffic? I can see a number of options. Remember: with SMB 3.0 traffic, the agent on the hosts causes VSS to create a coordinated VSS snapshot, and the backup server retrieves backup traffic from a permission controlled (Backup Operators) hidden share on the file server or SOFS (yes, your backup server will need to understand this).

Dual/Triple Homed Backup Server: The backup server will be connected to the server/VM networks. It will also be connected to one or both of the storage networks, depending on how much network resilience you need for backup, and what your backup product can do. A QoS (DCB) rule(s) will be needed for the backup protocol(s).
A dedicated backup NIC (team): A single (or teamed) physical NIC (team) will be used for backup traffic on the host and SOFS nodes. No QoS rules are required for backup traffic because it is alone on the subnet.
Create a backup traffic VLAN, trunk it through to a second vNIC (bound to the VLAN) in the hosts via the virtual switch. Apply QoS on this vNIC. In the case of the SOFS nodes, create a new team interface and bind it to the backup VLAN. Apply OS Packet Scheduler rules on the SOFS nodes for management and backup protocols.

With this design you get all the connectivity, isolation, and network path fault tolerance that you might have needed with 8 NICs plus fiber channel/SAS HBAs, but with superior storage performance. QoS is applied using DCB to guarantee minimum levels of service for the protocols over the rNICs.

In reality, it’s actually a simple design. I think people over think it, looking for a NIC team or protocol connection process for the rNICs. None of that is actually needed. You have 2 isolated networks, and SMB Multichannel figures it out for itself (it makes MPIO look silly, in my opinion).

The networking chapter of Windows Server 2012 Hyper-V Installation And Configuration Guide goes from the basics through to the advanced steps of understanding these concepts and implementing them:

Technorati Tags: Windows Server 2012,Hyper-V,Virtualisation,Storage,Failover Clustering

KB2836402 – You Cannot Add VHD/X Files To Hyper-V VMs On WS2012

Microsoft released a hotfix for when you cannot add VHD or VHDX files to Hyper-V virtual machines in Windows Server 2012.

Symptoms

Consider the following scenario:

You create some failover cluster nodes on computers that are running Windows Server 2012.

You have the Hyper-V server role installed on the cluster nodes.

You create virtual machines on one cluster node, and you configure the virtual machines as cluster resources.

You create multiple Cluster Shared Volume (CSV) resources and create one Virtual Hard Disk (VHD) file in each CSV.

You use Hyper-V Manager to try to add the VHD files to the virtual machines.

In this scenario, you cannot add the VHD files to the virtual machines. Additionally, you receive an error message that resembles the following:

Error applying Hard Drive changes
‘Virtual machine‘ failed to add resources to ‘virtual machine‘
Cannot add ‘C:ClusterStorageVolume3Test3.vhdx‘. The disk is already connected to the virtual machine ‘virtual machine‘. (Virtual machine ID virtual machine ID)
‘Virtual machine‘ failed to add resources. (Virtual machine ID virtual machine ID)
Cannot add ‘C:ClusterStorageVolume3Test3.vhdx‘. The disk is already connected to the virtual machine ‘virtual machine‘. (Virtual machine ID virtual machine ID)

Cause

This issue occurs because multiple CSV volumes have the same 0000-0000 serial number. Therefore, the VHD files on different volumes are recognized as the same file.

A supported hotfix is available from Microsoft.

Technorati Tags: Windows Server 2012,Hyper-V,Failover Clustering,Virtualisation,Storage

Below are the blog posts and articles that I have written for the Petri IT Knowledgebase over the past few weeks, covering topics like Hyper-V storage, Hyper-V snapshots, the Microsoft Cloud OS, and VMM 2012 SP1: