Tag: Windows Azure

Mapping The Microsoft Hybrid Cloud – Work In Progress

I am attempting to map out the infrastructure elements (not the app/dev elements) of the Microsoft hybrid cloud. This is a work in progress. If you spot any missing pieces then please comment and I will update.

You’ve heard terms like Cloud OS and hybrid cloud. What do they mean? I will attempt to map out the Microsoft hybrid cloud’s infrastructure-as-a-service (IaaS) ans software-as-a-service (SaaS) elements in this post.

The Hybrid Cloud

A private cloud is a single-tenant (but many users) service that is typically run on-premise. Note that there is a concept of a hosted private cloud; this is where a hosting company runs your single tenant infrastructure. An example of a private cloud is Hyper-V with elements of System Center (VMM, App Controller, Windows Azure Pack, etc) running in your data centre.

A public cloud is a hosted multi-tenant service that you do not own, but you consume services from. The perfect examples of this are Amazon Web Services (AWS) and Microsoft Windows Azure. The hosting company runs and hides the infrastructure from you. You subscribe to services from this shared infrastructure and have no visibility of other tenants. Those offerings are IaaS. There is platform-as-a-service (PaaS) which Windows Azure also offers for developers to run their applications without worrying about VM guest operating systems. And there is software-as-a-service (SaaS) such as Office 365 and Windows Intune where you use some software that the hosting company runs and sells to you from the cloud.

A hybrid cloud is where you mix elements of private cloud with public cloud. Microsoft is in a very unique position because they operate/sell IaaS, PaaS, and SaaS in public and private cloud. This allows you to integrate the best elements (for you) of on-premise with the public cloud offerings of Microsoft to create a hybrid offering.

The Map

image View the image to see full size

Windows Azure Site-Site VPN

You can deploy virtual machines in Windows Azure. They are very similar to Hyper-V VMs, because at this point, Windows Azure is running WS2012 Hyper-V (not WS2012 R2, as you can tell by digging around). You can deploy Software-Defined-Networking (SDN) within Windows Azure in the form of Virtual Networks; you define a network and then you define automatically routed subnets. You can configure a remote gateway to enable site-to-site VPN connectivity between your on-premise infrastructure and the network within Windows Azure. That creates intriguing possibilities where you run some services within Windows Azure to take advantage of elasticity and instant resource availability, and take advantage of on-premise where you can customise and specialise to your heart’s content.

An MPLS alternative has gone into beta with AT&T in the USA. Basically the Windows Azure network becomes another branch office on your WAN. That would be a much nicer and more fault tolerant option than single site-to-site VPN.

Note:

You will use SCVMM to manage your on-premise cloud(s) and use System Center App Controller to enable easy deployment of VMs/services in your hybrid cloud.

Active Directory

One of the biggest historical pains in IT for users is having multiple usernames and passwords. You can have single-sign-on (SSO) across your on-premise and Microsoft public cloud services by synchronising Active Directory with Windows Azure Active Directory (WAAD). WAAD is used in a couple of ways:

  • PaaS: Developers can use synchronised IDs for their custom applications.
  • SaaS: Office 365 (Midsize [M] plan and up) and Windows Intune can use the same user names for Exchange Online, SharePoint Online, Lync Online, etc, as are entered when users sign into their PC every day.

There are two ways to synchronise AD with WAAD:

  • DirSync: Is a simple-to-install and manage solution for smaller businesses.
  • ADFS: Active Directory Federation Services is used for larger installs. It requires HA because ADFS becomes a point of dependency to sign into services.

Another interesting option is to deploy VMs into Windows Azure, promote one or more to be domain controllers, and treat that as another site in your Active Directory forest. Your on-premise DCs will replicate with the DCs running in Windows Azure. This is used to enable traditional user & computer join/login to your AD forest.

Note: You must follow specific guidelines for creating DCs in Windows Azure. For example, all domain databases must be placed on an additional data drive that you attach to the VM. This is required to avoid corruption.

Office 365

I’ve already mentioned how users can sign into Office 365 (M plan and higher) using the same username and password as they use on their PC. You can also run hybrid Office services. For example, an Exchange organisation can span on-premise Exchange servers and the cloud.

Windows Intune & System Center Configuration Manager

System Center Configuration Manager (SCCM) is Microsoft’s corporate device deployment & management solution. I believe it is best used when limited to direct management of domain-joined Windows computers. Note that SCCM does allow you to deploy a distribution point (a content library that users/computers install from) in the cloud (hosted by Windows Azure).

You can also get Windows Intune, Microsoft’s cloud-based device management solution. Being cloud based makes it easy to deploy, and better for managing remote or widely distributed devices. Intune is less AD-centric, and that also makes it a great product for dealing with bring-your-own-device (BYOD). And Intune is also designed from the ground up to manage non-Windows OSs such as Android, iOS, and Windows Phone.

You can integrate Windows Intune into SCCM so admins have a single console to manage. I see Intune as the mechanism for dealing with widely distributed devices, roaming devices, mobile devices, and BYOD. SCCM is the solution for dealing with domain-joined corporate computers.

System Center Operations Manager

SCOM is Microsoft’s service-focused monitoring solution. You can get lots of Microsoft developed (free) management packs for monitoring on-premise stuff such as Windows Server, AD, SQL Server, and much more. There are also free third-party management packs (HP, Dell, Citrix, and more), and paid-for products from the likes of Veeam (which happens to have a limited free package for vSphere monitoring).

SCOM can also be used with the cloud in a few ways:

  • Global Service Monitor: GSM allows you to monitor the availability and quality of web services from Microsoft’s data centres around the world. This accounts for the fact that the Internet is complex and localised failures can affect international service availability in unpredictable ways. You configure GSM to monitor site(s) and the results appear in SCOM.
  • System Center Advisor: Think of this as a best practices analyzer from the cloud. SCOM can monitor the results of Advisor scans.
  • Windows Azure: You can monitor the services that you deploy in Azure in two ways. You can monitor the Azure service itself for failures. You can also install SCOM agents into the guest OS of your VMs to monitor the OS and services from within the VMs.

StorSimple

Many businesses struggle with retaining archive data. Microsoft acquired StorSimple to deal with that issue. This is a on-premise installed 1 GbE iSCSI storage appliance that offers local SSD and HDD tiers with a third colder tier residing within the storage services of Windows Azure.

The appliance is not suitable for all workloads. A key requirement is that your data must have a concept of a “working set”. In other words, there is hot data that you use frequently, and cold data that your do not look at very often. VM VHD/VHDX files are not examples of this. Think of a corporate file server, an CAD library, etc. Those are good examples.

StorSimple also has a built-in backup system that uses snapshot mechanisms to backup your hot/cold data.

Windows Azure Online Backup

There are many ways to use the storage mechanisms in Azure. Another one is to use Online Backup to automate the off-site storage of your backup data. A basic system for a single server would be to let Windows Server Backup send its data directly to the cloud. Larger customers might use something like System Center Data Protection Manager or Commvault Sympana to send their backup data to Windows Azure.

The data is encrypted using your private key. Microsoft never sees this key, and therefore you must keep the key safe; they cannot rescue you if you lose it.

I’ve been told that there is a beta in the USA to assist with getting that first big backup into the data center using secure out of band couriers. This will be a much more complex service to export due to the nature of international cross-border complexities.

Hyper-V Recovery Manager

HRM is not a solution that I am convinced about, due to pricing and the fact that it lives in Azure. I prefer micro-payment and placement in the secondary site.

However, HRM is an orchestration solution that lives in Windows Azure for coordinating Hyper-V Replica between two VMM-managed Hyper-V sites. Asynchronous replication data flows directly between the two sites, never to Azure. HRM purely manages replication and failover.

SQL Server 2014

SQL  Server AlwaysOn availability groups can span on-premise and in-Azure VMs, enabling hybrid cloud HA of your relational data services.

Using WatchGuard XTM To Create A Hybrid Cloud With Windows Azure

My job is weird. I basically get told to learn something and spend time promoting it, teaching it, assisting with it to a Microsoft partner audience in Ireland. Lately we’ve taken on some hardware products and I’ve also been given a target to promote Windows Azure. So I’ve been spending time in the lab at work and in Windows Azure.

The latest “mini project” that I set for myself was to create a hybrid cloud, merging my on-premise Hyper-V farm (with SMB 3.0 storage on DataOn Storage JBOD) with VMs running in Windows Azure. Traffic between the two “sites” would be via a secure site-site VPN tunnel. This is Microsoft’s strategy: hybrid cloud.

The On-Premise VPN Concentrator

The first step in that was to get a new firewall appliance operational. Although you can use an on-premise Windows Server to create a site-site VPN connection, I don’t like that option. I’d rather use an edge appliance so my routing can be simplified.

Note: I’m documenting my experience instead of the specific instructions. You’ll read why later.

My employers recently started distributing the XTM range of universal threat management (UTM) firewall appliances from WatchGuard to the Irish reseller market. I have an old 2 series appliance in my lab, equipping me with firewall, AV, URL management, wireless and VPN connectivity. While the hardware might be old, it’s running the latest software and management interface and gives me all the same functionality as the latest and largest 8 series appliances from WatchGuard (just with smaller scalability).

WatchGuard 2 series XTM

I placed the WatchGuard behind the Netgear ADSL router, and have enabled ports passthrough from the router to the firewall:

  • L2TP port: UDP 1701
  • IPsec port: UDP 500
  • IKEv2 port: UDP 4500

My internal network is physical, operating on 172.16.1.0/24, with the XTM being the default gateway on 172.16.1.1.

Enabling Site-Site VPN in Windows Azure Virtual Networking

The next thing I did was sign into Windows Azure and create a virtual network. It’s not quite obvious, but what you are doing in the Azure portal is creating software-defined networks using Hyper-V Network Virtualization. I created a virtual network called 10.0.0.0/16 and then created 3 virtual subnets:

  • 10.0.0.0/24
  • 10.0.1.0/24
  • 10.0.2.0/24

Any virtual machines I created would reside in those subnets and be assigned IPs from those pools (they appear like DHCP addresses in the guest OS). Note that Azure uses a few of the IPs in each virtual subnet and that the subnets will route automatically to each other within the virtual network.

An additional gateway subnet was created on 10.0.255.0/24.

image

My virtual network and subnets in Windows Azure

Here’s the fun bit; you can assign IP address(es) for your desired DNS server(s) in the virtual network settings. I assigned 172.16.1.40, my on-premise DC/DNS VM, as the DNS server for this in-Azure virtual network. My plan: I would only run DCs on premise, and everything in Azure will authenticated against my on-premise DCs via the VPN. Honestly, in the real world I think I would run some VMs as DCs in the same domain/forest within Azure for network fault tolerance. Old fashioned AD replication would be used, treating Azure’s virtual network as another AD site.

During the virtual network wizard, I also enabled site-site connectivity and afterwards I created a gateway. That creates the listener in Azure, on a public IP address, that allows a site-site VPN connection. A really long secret key is created.  I documented all the required information and then returned to the lab.

Starting & Testing The Site-Site VPN

I logged into the console for the WatchGuard XTM and created a site-site VPN connection. The connection was initiated, and then there was suspense. In the Azure portal I could see an “attempting connection” status. That sat there for what felt like an eternity. And then … bingo! It connected.

image

The connected site-site VPN, details obscured

I fired up a VM in Windows Azure on my 10.0.0.0/24 network. It was assigned the first address, 10.0.0.4 with the DNS setting pointing to my DC which is on-premise as 172.16.1.40. With the Windows Firewall configured for ICMPv4 echo requests, I was able to ping in both directions.

The end result? The virtual network in Windows Azure is effectively a remote data center in my “corporate network”. My on-premise 172.16.1.0/24 can route to the 10.0.0.0/16 network/subnets in Windows Azure and back again. I can deploy VMs to the most suitable networks: on-premise or in the public cloud. If I fire up System Center VMM and App Controller, I can delegate users and give them a single portal for deploying VMs on either part of the hybrid cloud.

Some Useful Info

I had two sources of information to implement this solution.

The first was the excellent blog post by Ryan Boud called Creating a VPN between a WatchGuard XTM 510 and Windows Azure Virtual Networks. The terminology for setting up the site-site VPN is confusing: What’s a local subnet? What’s a remote subnet? It’s all relative! Ryan has excellent clear screenshots that inform you what to type where in the Windows Azure portal to create your virtual network and get the gateway operational. He also goes step-by-step through the WatchGuard XTM configuration.

The second is a set of instructions by WatchGuard. Their documentation only covers the XTM side of things but it does give you a nice method for recording the required information from the Azure portal.

Microsoft has also  shared links to instructions for creating site-site VPN connections using devices from lots of manufacturers, such as Cisco, Juniper, F5, Citrix, Fortinet and Openswan.

FYI, my lab is operating on an ADSL line. It has a single IP address. I am still able to do remote device VPN into my lab. In fact, I am able to VPN into the lab from home and communicate with the Windows Azure VMs by routing through the site-site VPN connection. The Windows Azure network is really acting like a remote data center for my lab.

Summary

I thought setting the site-site VPN up between my “private cloud” and Microsoft’s public cloud was going to be a nightmare. Instead, it was easy. In fact, following Ryan’s and WatchGuard’s instructions enabled me to get it working on my first attempt. The results: magic.