Azure Backup | Aidan Finn, IT Pro

Create a New VM From An Existing Managed Disk

In previous posts I have shown how to restore the disks of a VM to a storage account and how to create managed disks from those VHD blobs. In this post, I will show how to create a new VM from a managed disk. When these 3 steps are done together, this is an easy way to restore an Azure virtual machine from backup to an availability set.

I previously created a managed disk from a restored VHD blob, and stored it in a resource group called demorestore. I deliberately named the new managed disk after the VM that I am going to create.

You can only create a new VM from a managed disk that contains an operating system. In the below screenshot, you can see that this disk contains Windows. If this is an OS disk, then you can click the magic button called + Create VM.

What you are doing by clicking the button is shorting the usual Create Virtual Machine blade/wizard. A blade you probably know appears, but some of the features are greyed out because they’re already selected by choosing to create a VM from an existing managed disk.

Enter the name of the new VM, and select the resource group.

In the Size blade, choose the size of the new VM. In settings, choose the availability set (key to restoring a VM to an availability set) and then all the other stuff like network, subnet, extensions, etc.

When you complete the wizard, a VM (which is just metadata) is created using your pre-existing OS managed disk. If you have any data disks to re-use, open Disks in the settings of the VM and add those managed disks with the required host caching mode. And that’s all there is to it!

Create an Azure Managed Disk from a VHD Blob

This post will show you how to create a managed disk from a VHD blob file, such as one you’ve uploaded or restored from a virtual machine backup. In my example, I have restored the virtual hard disks of an Azure VM to a storage account called aidanfinnrestore. I am going to create a new managed disk from the VHD blob, and (in another post) create a new VM from the managed disk that I am creating in this post.

Open the Azure Portal, and go to Disks in the navigation bar on the left – this is where all managed disks are listed. Click + Add. A Create Manage Disk blade appears. Enter the following information:

Name: Give the new managed disk a name. My naming standard names the disk after the VM with a suffix to denote a role. In my example, it’s an OS disk.
Subscription: Select the subscription in your tenant. Note that you must create the managed disk in the same subscription as the storage account that contains the blob – you can always move the disk to a different subscription later.
Resource Group: Restore the disk to a new or existing resource group – typically this is where the virtual machine will be.
Location: Pick the region of the desired VM, which must also match the storage account.
Account Type: What kind of managed disk do you want – Standard (HDD) or Premium (SSD). You can change this later, one of the nice features of managed disks.
Source Type: I have selected Storage Blob – this is how the restored (or uploaded) VHD is stored.
Source Blob: Click browse, and navigate to & select the VHD blob that was restored/uploaded.
OS Type: If this is a data disk then select either Windows or Linux, depending on the guest OS in the VHD.
Size: To make like easy, select the size of the existing blob. I restored a managed disk to a blob, so I went with the original size of 128 GiB.

Once you’re happy with all the settings, click Create. In my case, with a 128 GiB VHD, the creation just around 30 seconds:

Now you can either create a VM from the disk or attach it as a data disk to an existing VM in the Azure Portal – life is easy with managed disks!

Restore an Azure Virtual Machine’s Hard Disks

In this post, I’ll show you how to restore just the disks of an Azure virtual machine. This is useful if you want to restore a virtual machine to an availability set, or restore it as a different series/size.

Restoring to Availability Sets

For some reason that I do not know, we cannot restore a virtual machine to an availability set in Azure. It probably has something to do with the restriction in ARM that prevents a VM from being able to join an availability set after creation (vote for change here).

As a workaround, Azure Backup allows you to restore the disks, and then use those disks to create a new virtual machine (metadata) that is joined to the availability set. On the official docs pages, there is some pretty messy looking PowerShell to re-create the VM from those disks.

Thanks to some features of Managed Disks, if you have used managed disks for the VM, then you don’t need to go anywhere near that nasty PowerShell or JSON! I’ll post about that soon.

Restoring Disks

Browse to to the recovery services vault, open it, go to Backup Items > Azure Virtual Machine, and select the VM in question. Below is a screenshot of my web server in Azure. Click Restore VM.

A blade with recovery points appears. Choose a restore point, i.e. a point in time from when you want to restore from, and click OK.

The Restore Configuration blade appears. Choose Restore Disks as the Restore Type, and choose a storage account as the Staging Location. Click OK to start the backup job.

Some time later, the disk(s) of the virtual machine are restored as blobs in a container in the storage account. You’ll also find a JSON file with details of the disk(s) that were restored.

By the way, if you cannot tell which of the VHD blobs is your OS disk, download the JSON file and open it in Notepad (VS Code refuses to open it for me). The “osDisk” setting will tell you the path of the VHD blob that was the original data disk.

Microsoft’s solution would have you restore the virtual machine using PowerShell and that JSON file. I’ve read through it – it’s not pretty! My solution, in a later post, would create managed disks from the VHD blob(s), and then create a VM from the OS disk … and that’s nice and easy using the Azure Portal and a few mouse clicks.

How I Upgraded This VM To Azure Resource Manager

In this post, I will explain how I upgraded the Azure virtual machine, that this site is hosted on, from ASM (Azure Service Management, Classic or Azure v1) to ARM (Azure Resource Manager, Resource Manager or Azure v2).

FYI, although I had no plans for changing subscriptions, this migration is also the first step for moving resources from one subscription to another, e.g. Credit Card/Direct to CSP, Open to CSP, or EA to CSP.

Background

I’ve been running aidanfinn.com on Azure for a few years now. It game me a vested interest in Azure, other than my day job where I teach and sell Azure services to MS partners. Over the years, I’ve applied some of the things that I’ve learned, including one time where MySQL blew up so bad that I had to use Azure Backup (then in preview) to restore the entire VM – this is why you’ll see the phrase “aidanfinn02” later in the example.

The VM was running in ASM. ASM is effectively deprecated, but I never had a chance to migrate it to ARM. That changed, and I decided to make the switch. In the process, I decided that I wanted to clean up resource groups, migrate to managed disks, and maybe change a few other things.

I had two options available to me:

MigAz: Great community tool that allows a complete change, including renaming. There is downtime to move the disks.
The “Platform Supported” method: Using official cmdlets to do an ASM-ARM migration with no downtime during the ASM-ARM migration.

I went with the platform supported method. Long-term, it means I have more work to do to do some renaming, etc, but I’ve never had “my own” stuff to move using this method, so I wanted to do it and document it with a real example.

Note that the platform support method has a few approaches. My virtual machine was in a virtual network, so I opted to move the entire virtual network and associated contents. This is cool because, all VMs (I have one only) are moved, and endpoints are converted into NAT rules in a load balancer. Any reserved cloud service IPs are converted into static public IPs.

Note, you’ll need to download the latest Azure PowerShell modules (and reboot if it’s your first install) to do either method.

FYI, the below includes copy/pastes of the actual cmdlets that I used. The only thing I have modified, for obvious reasons, is the subscription ID.

Register the Migration Provider

You’ll need to register the provider that allows you to do ASM-ARM migrations. This is done on a per subscription basis. You’ll log into your subscription using ARM:

Login-AzureRmAccount

If your tenant has more than one subscription, like mine does, then you need to query the subscriptions:

Get-AzureRmSubscription

This allows you to get the ID of the subscription that you will sign into, so you can select it as the current subscription to work on:

Select-AzureRmSubscription -SubscriptionId 1234567f-a1b2-1234-1a2b-1234ab123456

Next you register the migration proivder:

Register-AzureRmResourceProvider -ProviderNamespace Microsoft.ClassicInfrastructureMigrate

This migration can take up to five minutes. You can query the status of the migration:

Get-AzureRmResourceProvider -ProviderNamespace Microsoft.ClassicInfrastructureMigrate

You can continue once you get a registered status:

ProviderNamespace : Microsoft.ClassicInfrastructureMigrate

RegistrationState : Registered

ResourceTypes : {classicInfrastructureResources}

Locations : {East Asia, Southeast Asia, East US, East US 2…}

Validation

The migration has three phases. The first allows you to validate that you can migrate your machines cleanly. Common issues include unsupported extensions, extension versions, classic backup vault registrations, endpoint ACLs, etc.

To do the validation, you need to sign in using ASM:

Add-AzureAccount

Once again you need to make sure you’re working with the right subscription:

Get-AzureSubscription

And then make that subscription current:

Select-AzureSubscription -SubscriptionId 1234567a-a1b2-1234-1a2b-1234ab123456

I migrated my virtual network, so I queried for the Azure name of the virtual network.

Get-AzureVnetSite | Select -Property Name

My virtual network was called aidanfinn02 so I saved that as a variable – it’ll be used a few times.

$vnetName = “aidanfinn02”

Then I ran the validation against the virtual network, saving the results in a variable called $validate:

$validate = Move-AzureVirtualNetwork -Validate -VirtualNetworkName $vnetName

I could then see the results:

$validate.ValidationMessages

There were two issues:

A faulty monitoring (Log Analytics) extension (guest OS agent)
The VM was being backed up by a classic Azure Backup backup vault.

ResourceType : VirtualNetwork

ResourceName : aidanfinn02

Category : Error

Message : Virtual Network aidanfinn02 has encountered validation failures, and hence it is not supported for migration.

VirtualMachineName :

ResourceType : Deployment

ResourceName : aidanfinn02

Category : Error

Message : Deployment aidanfinn02 in Cloud Service aidanfinn02 has encountered validation failures, and hence it is not supported for migration.

VirtualMachineName :

ResourceType : Deployment

ResourceName : aidanfinn02

Category : Error

Message : VM aidanfinn02 in HostedService aidanfinn02 contains Extension MicrosoftMonitoringAgent reporting Status : Error. Hence, the VM cannot be migrated. Please ensure that

the Extension status being reported is Success or uninstall it from the VM and retry migration.,Additional Details: Message=This machine is already connected to another

Log Analytics workspace, please set stopOnMultipleConnections to false in public settings or remove this property, so this machine can connect to new workspaces, also

it means this machine will get billed multiple times for each workspace it report to. (MMAEXTENSION_ERROR_MULTIPLECONNECTIONS) Code=400

VirtualMachineName : aidanfinn02

ResourceType : Deployment

ResourceName : aidanfinn02

Category : Error

Message : VM aidanfinn02 in HostedService aidanfinn02 contains Extension MicrosoftMonitoringAgent reporting Handler Status : Unresponsive. Hence, the VM cannot be migrated.

Please ensure that the Extension handler status being reported is Ready or uninstall it from the VM and retry migration.,Additional Details: Message=Handler

Microsoft.EnterpriseCloud.Monitoring.MicrosoftMonitoringAgent of version 1.0.11049.5 is unresponsive Code=0

VirtualMachineName : aidanfinn02

ResourceType : Deployment

ResourceName : aidanfinn02

Category : Error

Message : VM aidanfinn02 in HostedService aidanfinn02 is currently configured with the Azure Backup service and therefore currently not supported for Migration. To migrate this

VM, please follow the procedure described athttps://aka.ms/vmbackupmigration.

VirtualMachineName : aidanfinn02

The solutions to these problems were easy:

I signed into the classic Azure Management Portal and unregistered the virtual machine in the backup vault – DO NOT DELETE THE BACKUP DATA!
Then I switched to the Azure Portal (and stayed here), and I removed the VMSnapshot (Azure Backup) extension from the VM.
And then I removed the Microsoft.EnterpriseCloud.Monitoring (Log Analytics) extension
The VM had an old version of the Diagnostics agent, so I removed that too, even though it didn’t effect validation.

I re-ran the validation:

$validate = Move-AzureVirtualNetwork -Validate -VirtualNetworkName $vnetName$validate.ValidationMessages

And the result was that the virtual network (and thus the VM) was ready for an ARM migration.

$validate.ValidationMessages

ResourceType : VirtualNetwork

ResourceName : aidanfinn02

Category : Information

Message : Virtual Network aidanfinn02 is eligible for migration.

VirtualMachineName :

ResourceType : Deployment

ResourceName : aidanfinn02

Category : Information

Message : Deployment aidanfinn02 in Cloud Service aidanfinn02 is eligible for migration.

VirtualMachineName :

ResourceType : Deployment

ResourceName : aidanfinn02

Category : Information

Message : VM aidanfinn02 in Deployment aidanfinn02 within Cloud Service aidanfinn02 is eligible for migration.

VirtualMachineName : aidanfinn02

Preparation

The preparation phase is next. This is an interim or trial period where you introduce the ARM API to your resources. For a time, the resources are visible to both ARM and ASM. There is no downtime, but you cannot make any configuration changes. The reasoning for this is that you can validate that everything is still working.

You run the command, as I did against the virtual network.

Move-AzureVirtualNetwork -Prepare -VirtualNetworkName $vnetName

And then you wait … try not to grind your teeth or chew your gums:

OperationDescription OperationId OperationStatus

——————– ———– —————

Move-AzureVirtualNetwork 2bcb56ce-2330-0824-a376-a3dcc4892e3d Succeeded

Commit

Once preparation is done, double-check everything. My website was still responding, and resources appeared in two different resource groups with -Migrated suffixes – I’ll show you how I tidied that up later in the post. I was ready to commit; this is when you tell Azure that all is good, and to please remove the ASM APIs from the resources.

Move-AzureVirtualNetwork -Commit -VirtualNetworkName $vnetName

More time passes when you forget to breath, and then it’s done! You’re in ARM, and your resources are manageable once again. Note that there is an alternative to Commit, which is to abort the process and roll back to ASM-only management.

Network Security Group

My ASM deployment was quite basic, and not best practice. I created a network security group for the subnet, allowing in:

RDP to the subnet
HTTP to the local static IP of the VM.

Resource Group Clean-up

As I mentioned, my ASM resources migrated into two resource groups as ARM resources. I wanted to:

Move the resources into a single resource group.
Get rid of the –Migrated suffix.

You can move resources, but you cannot rename resource groups. So I created a third resource group (aidanfinn):

Then I moved the resources from both of the migrated resource groups into the new aidanfinn resource group:

Migrate Storage to ARM

The official method for migrating storage is to migrate the storage account. That means that you move the storage account with the VHDs within it. Azure offers a new method for handling storage called managed disks:

You dispense with the storage account for VHDs
The disk becomes a manageable resource in the Portal
You get cool new features like Snapshots and easier VM restores from disk

I decided to take a different route – I would convert my ARM VM from un-managed disks to managed disks. I needed a few values:

$rgName = "aidanfinn"
$vmName = "aidanfinn02"

I stopped the VM (my first piece of downtime in this entire process):

Stop-AzureRmVM -ResourceGroupName $rgName -Name $vmName -Force

Then I did the conversion:

ConvertTo-AzureRmVMManagedDisk -ResourceGroupName $rgName -VMName $vmName

And then I restarted the VM, after just a few minutes of downtime:

Start-AzureRmVM -ResourceGroupName $rgName -Name $vmName

Re-Introduce Management

I like the management features of Azure, so I re-introduced:

Azure Backup of the VM using a recovery services vault using a custom policy – I created a manual backup immediately.
Monitoring & diagnostics, to a new dedicated storage account – make sure you verify that the storage account is being used by the VM Agent and Boot Diagnostics.

Cleanup ASM

Lots of stuff can get left behind, especially if you’ve been trying things out. That all needs to be removed. One thing I kept around for a while was the classic backup vault, just in case. I only unregistered the old ASM VM – I did not delete the data. That means I have a way back if all goes wrong in ARM or I screw up in some way. I’ll give it a month, and then I’ll remove the old vault.

By the way, you can upgrade a backup vault to ARM (recovery services vault) if you want to keep your retention.

The End Result

I ended up with everything in ARM and in one resource group:

I don’t like the naming, so I will be cleaning things up. My next mini-project will be to:

Power down the VM.
Create a disk snapshot.
Create a new disk from the snapshot.
Create a new deployment, using the new disk, with names that I like.

I know – I could have done all that more quickly and easily using MigAz, but I wanted to do the platform supported migration on my stuff, and it was a chance to document it too! Hopefully this will be useful for you.

Azure Backup Central Reporting – Pay Attention MS Partners!

Microsoft has launched a preview of Azure Backup Reporting; this is a solution where you can export backup data to Power BI, and this allows you to consume, visualize, and subscribe to information about backup from many recovery services vaults in many subscriptions.

The way the system works is that you configure the recovery services vault to export data regularly to a storage account (must be in the same tenant as the recovery services vault).

You then sign into Power BI (a free subscription can be used but this is limited to 1 GB of data) and import the Azure Backup content pack.

Data is exported as JSON files into a folder (container) in the storage account, and Power BI will consume/process that data. The timing of this varies on the data, but Microsoft advises that it can take 24 hours for your first data sets to be consumed.

The default screens show lots of useful information:

Summary of job health
Cloud storage usage
Quantities of instances
Cloud storage growth trends

While the solution is not perfect yet (read more and vote here) it can be used today. Note that DPM, MABS, and Azure VM backup are not supported yet by the preview. I have set up 3 demo subscriptions (each in a different tenant as is normal for deployments by MS partners), each with a MARS backup job. I imported the Azure Backup content pack 3 times, 1 for each tenant. I made a custom report for each subscription and pinned them to a single dashboard. Now I can see the results of each and every backup job in one screen. I can also create daily/weekly email subscriptions to each report – that means I can send out these reports to my customers!

I can also publish the reports either to a web site or a private SharePoint (including Online) site – here’s an example that I did for work.

The end result is that we finally have a centralized reporting solution for Azure Backup. With one quick scroll, I can easily see the health of all of my customers’ backups.

Microsoft Azure Backup Server v2 Launched

Microsoft has launched version 2 of MABS, the Microsoft Azure Backup Server v2, with support for Windows Server 2016 and vSphere 6.5.

So far we’ve had 2 versions (v1 and v1 update 1) of MABS, the freely licensed (but your pay Azure Backup pricing) slightly modified version of System Center Data Protection Manager. MABS v1 was based on DPM 2012 R2, and MABS v2 is based on DPM 2016, with the cool features of DPM 2016:

Modern Storage, which improves performance and reduces consumption by leveraging ReFS Block Cloning, VHDX, and Deduplication.
Improves Hyper-V backup, by supporting WS2016 hosts and by using the built-in (WS2016 Hyper-V) Resilient Change Tracking (RCT) for incremental backups without 3rd party software being placed into the kernal of the host’s management OS.
Support for Shielded Virtual Machines, the ultra-secure platform on WS2016 Hyper-V.
Support for Storage Spaces Direct (S2D).
The ability to install MABS v2 on WS2016.

MABS v1 Update 1 added support for VMware vCenter & ESXi 5.5 and 6.0. MABS v2 adds vCenter & ESXi 6.5 to the list. Note that if you install MABS v2 on WS2016 then VMware protection will be in preview mode, while we wait for VMware to release support for VDDK 6.5 for WS2016. You can learn more on from this video.

You can download MABS v2 from here or from a recovery services vault in the Azure Portal.

The supported backup server configuration is:

Windows Server 2012 R2, Windows Server 2016
Processor: Minimum: 1 GHz, dual-core CPU. Recommended: 2.33 GHz quad-core CPU
RAM: Minimum: 4GB. Recommended: 8GB
Hard Drive Space (program files): Minimum: 3GB, Recommended: 3GB
Disks for backup storage pool: 1.5 times size of data to be protected

Microsoft Azure Backup MARS Agent Supports System State

Microsoft has announced that the Azure Backup MARS agent will support the protection of System State on Windows Server. This is a preview release.

I started talking about Azure Backup 3 years ago, and one of the “we’re not doing it” questions was “does it backup system state”. The answer was no. Azure Backup listened and now you can backup your system state to Azure using the MARS agent.

Scenarios discussed by the Azure Backup team include Active Directory, file server configurations, and IIS server configurations, where restoring files & folders is not enough; the metadata that makes those files & folders useful is stored in System State so the ability to restore that meta data is also important.

Supported versions of Windows Server in this preview release are:

W2008 R2
WS2012
WS2012 R2
Windows Server 2016

Do you want support for Windows Server 2003? Let me sell you some Ace of Base and Vanilla Ice cassettes!

This is good news, a part of the continuous improvement of Azure Backup driven by your feedback.

My Top 5 Features in System Center Data Protection Manager 2016

Microsoft’s System Center Data Protection Manager (DPM) has undergone a huge period of transition over the past two years. Significant investments have been made in hybrid cloud backup solutions, and DPM 2016 brings many improvements to this on-premises backup solution that all kinds of enterprise customers need to consider. Here are my top 5 features in DPM 2016.

5: Upgrading a DPM production server to 2016 doesn’t require a reboot

Times have changed and Windows Server & System Center won’t be released every 3-5 years anymore. Microsoft recognizes that customers want to upgrade, but fear the complexity and downtime that upgrades often introduce. Upgrading DPM servers and agents to 2016 will not cause production hosts to reboot.

4: Continued protection during cluster aware updates

The theme of continued protection during upgrades without introducing downtime continues. I’ve worked in the hosting business where every second of downtime was calculated in Dollars and Euros. Cluster-aware updates allow Hyper-V clusters to get security updates and hotfixes without downtime to applications running in the virtual machines. DPM 2016 supports this orchestrated patching process, ensuring that your host clusters can continue to be stable and secure, and your valuable data is protected by backup.

3: Modern Backup Storage

Few people like tapes, first used with computers in 1951! And one of the big concerns about backup is the cost of storage. Few companies understand software-defined storage like Microsoft, leading the way with Azure and Windows Server. DPM 2016 joins the ranks by modernizing how disk storage is deployed for storing backups. ReFS 3.0 block cloning is used to store incremental backups, improving space utilization and performance. Other enhancements including growing/shrinking storage usage based on demand, instead of the expensive over-allocation of the past.

2: Support for Storage Spaces Direct

While we’re discussing modern storage, let’s talk about how DPM 2016 has support for Microsoft’s software-defined hyper-converged infrastructure solution, Storage Spaces Direct. In recent years, these two concepts, inspired by the cloud, have shaken up enterprise storage:

Software-defined storage: Customers have started to realize that SAN isn’t the best way to deploy fast, scalable, resilient, and cost-effective storage. Using commodity components, software can overcome the limitations of RAID and the expense of proprietary lock-in hardware.
Hyper-converged infrastructure: Imagine a virtualization deployment where there is one tier of hardware; storage and compute are merged together using the power of software and hardware offloads (such as SMD Direct/RDMA), and turn cluster deployments into a simpler and faster process.

Windows Server 2016 took lessons from the previous two versions of Storage Spaces, Azure, and the storage industry and made hyper-converged infrastructure a feature of Windows Server. This means that you can deploy an extremely fast (NVMe, SSD, and HDD disks with 10 Gbps or faster networking) storage that is cost effective, using 1U or 2U servers, and with no need for a SAN, external SAS hardware, or any of those other complications. DPM 2016 supports this revolutionary architecture, ensuring the protection of your data on the Microsoft on-premises cloud.

1: Built for the Cloud

I’ve already discussed the cost of storage, but that cost is doubled or more once we start to talk about off-site storage of backups or online-backup solutions. While many virtualization-era backup products are caught up on local backup bells and whistles, Microsoft has transformed backup for the cloud.

Combined with Azure Backup, DPM 2016 gives customers a unique option. You get enterprise-class backup that protects workloads on cost effective (Modern Backup Storage) storage for on-premises short term retention. Adding the very affordable Azure Backup provides you with a few benefits, including:

A secondary site, safeguarding your backups from localized issues.
Cost effective long-term retention for up to 99 years.
Encrypted “trust no-one” storage with security mechanisms to protect you against ransom-ware and deliberate attacks against your backups.

In my opinion, if you are not using DPM, or have not looked at it in the past two years, then I think it’s time to re-evaluate this product.

Seeding Azure Backup Using Secure Disk Transfer

Microsoft’s online backup service, Azure Backup, was recently updated to greatly improve how the first big backup is done to the cloud. These improvements impacted the Azure Backup MARS agent, Microsoft Azure Backup Server, and System Center Data Protection Manager (DPM). I recently recorded a short video to explain the problem, the soluition, and I show how you can use it – the process is the same across each of the 3 products.

Technorati Tags: Azure,Azure Backup,Backup,System Center,DPM