Azure Virtual WAN – Connectivity

In this post, I’ll explain how Azure Virtual WAN offers its core service: connections.

SD-WAN

Some of you might be thinking – this is just for large corporations and I’m outta here. Don’t run just yet. Azure Virtual WAN is a rethinking of how to:

  • Connect users to Azure services and on-premises at the same time
  • Connect sites to Azure and (optionally) other sites
  • Replace the legacy hardware-defined WAN
  • Connect Azure virtual networks together.

That first point is quite timely – connecting users to services. Work-from-home (WFH) has forced enterprises to find ways to connect users to services no matter where they are. That connectivity was often limited to a privileged few. The pandemic forced small/large organisations to re-think productivity connectivity and to scale out. Before COVID19 struck, I was starting to encounter businesses that were considering (some even starting) to replace their legacy MPLS WAN with a software-defined WAN (SD-WAN) where media of different types, suitable to different kinds of sites/users/services, were aggregated via appliances; this SD-WAN is lower cost, more flexible, and by leveraging local connectivity, enables smaller locations, such as offices or retail outlets, to have an affordable direct connection to the cloud for better performance. How the on-premises part of the SD-WAN is managed is completely up to you; some will take direct control and some will outsource it to a network service provider.

Connections

Azure Virtual WAN is all about connections. When you start to read about the new Custom Routing model in Azure Virtual WAN, you’ll see how route tables are associated with connections. In summary, a connection is a link between an on-premises location (referred to as a branch, even if it’s HQ) or a spoke virtual network with a Hub. And now we need to talk about some Azure resources.

Azure Resources

I’ve provided lots more depth on this topic elsewhere so I will keep this to the basics. There are two core resources in Azure Virtual WAN:

  • A Virtual WAN
  • A Hub

The Virtual WAN is a logical resource that provides a global service, although it is actually located in one Azure region. Any hubs that are connected to this Virtual WAN resource can talk to each other (automatically), route it’s connections to another hub’s connections and share resources.

A Virtual WAN Hub is similar to a hub in an Azure hub & spoke architecture. It is a central routing point (with a hidden virtual router) that is the meeting point for any connections to that hub. An Azure region can have 1 hub in your tenant. That means I can have 1 Hub in West Europe and 1 Hub in East US. The Hubs must be connected to an Azure WAN resource; if they share a WAN resource then their connections can talk to each other. I might have all my branches in Europe connect to the Hub in West Europe, and I will connect all my spoke virtual networks in West Europe to the Hub in West Europe too; this means that by default (and I can control this):

  • The virtual networks can route to each other
  • The virtual networks can route to the branches
  • The branches can route to the virtual networks
  • The branches can route to other branches

We can extend this routing by connecting my branches in North America to the East US Hub and the spoke virtual networks in East US to the East US Hub. Yes; all those North American locations can route to each other. Because the Hubs are connected to a common Virtual WAN, the routing now extends across the Microsoft WAN. That means a retail outlet in the further reaches of northwest rural Ireland can connect to services hosted in East US, via a connection to the Hub in West Europe, and then hopping across the Atlantic Ocean using Microsoft’s low-latency WAN. Nice, right? Even, better – it routes just like that automatically if you are using SD-WAN appliances in the branches.

Note that a managed WAN might wire up that retail outlet differently, but still provide a fairly low-latency connection to the local Hub.

Branch Connections

If you have done any Azure networking then you are probably familiar with:

  • Site-to-site VPN: Connecting a location with a cost-effective but no-SLA VPN tunnel to Azure.
  • ExpressRoute: A circuit rented from an ISP for low-latency, high bandwidth, and an SLA-supported private connection to Azure
  • Point-to-Site VPN: Enabling end-users to create a private VPN tunnel to Azure from their devices while on the move or working from home

Each of the above is enabled in Azure using a Virtual Network Gateway, each running independently. Routing from branch to branch is not an intended purpose. Routing from user to the branch is not an intended purpose. The Virtual Network Gateway’s job is to connect a user to Azure.

The Azure Virtual WAN Hub supports gateways – as hidden resources that must be enabled and configured. All three of the above media types are supported as 3 different types of gateway, sized based on a billing concept called scale units – more scale units means more bandwidth and more cost, with a maximum hub throughput of 40 Gbps (including traffic to/from/between spokes).

Note that a Secured Virtual Hub, featuring the Azure Firewall, has a limit of 30 Gbps if all traffic is routed through that firewall.

You can be flexible with the branch connections. Some locations might be small and have a VPN connection to the Hub. Other locations might require an SLA and use ExpressRoute. Some might require low latency or greater bandwidth and use higher SKUs of ExpressRoute. And of course, some users will be on the move or at home and use P2S VPN. A combination of all 3 connection types can be used at once, providing each location and user the connections and costs that suit them best.

ExpressRoute

You will be using ExpressRoute Standard for Azure Virtual WAN; this is a requirement. I don’t think there’s really too much more to say here – the tech just works once the circuit is up, and a combination of Global Reach and the any-to-any connections/routing of Azure WAN means that things will just work.

Site-to-Site VPN

The VPN gateway is deployed in an active/active cluster configuration with two public IP addresses. A branch using VPN for connectivity can have:

  • A single VPN connection over a single ISP connection.
  • Resilient VPN connections over two ISP connections, ideally with different physical providers or even media types.

An on-premises SD-WAN appliance is strongly recommended for Azure Virtual WAN, but you can use any VPN appliance that is supported for route-based VPN by Microsoft Azure; if you are doing the latter you can use BGP or the Azure WAN alternative to Local Network Gateway-provided prefixes for routing to on-premises.

Point-to-Site (P2S) VPN

The P2S gateway offers a superior service to what you might have observed with the traditional Virtual Network Gateway for VPN. Connectivity from the user device is to a hub with a routing appliance. Any-to-any connectivity treats the user device as a branch, albeit in a dedicated network address space. Once the user has connected the VPN tunnel, they can route to (by default):

  • Any spoke virtual network connected to the Hub
  • Any spoke virtual network connected to another Hub on the same Virtual WAN
  • Any branch office connected to any Hub on the Virtual WAN

In summary, the user is connected to the WAN as a result of being connected to the Hub and is subject to the routing and firewall configurations of that Hub. That’s a pretty nice WFH connectivity solution.

Note that you have support for certificate and RADIUS authentication in P2S VPN, as well as the OpenVPN and Microsoft client.

The Connectivity Experience

Imagine we’re back in normal times again with common business travel. A user in Amsterdam could sit down at their desk in the office and connect to services in West Europe via VPN. They could travel to a small office in Luxembourg and connect to the same services via VPN with no discernible difference. That user could travel to a conference in London and use P2S VPN from their hotel room to connect via the Amsterdam Hub. Now that user might get a jet to Philadelphia, and use their mobile hotspot to offer connectivity to the Azure Virtual WAN Hub in East US via P2S VPN – and the experience is no different!

One concept I would like to try out and get a support statement on is to abstract the IP addresses and locations of the P2S gateways using Azure Traffic Manager so the user only needs to VPN to a single FQDN and is directed (using the performance profile) to the closest (latency) Hub in the Virtual WAN with a P2S gateway.

Simplicity

So much is done for you with Azure Virtual WAN. If you like to click in the Azure Portal, it’s a pretty simple set up to get things going, although security engineering looks to have a steep learning curve with Custom Routing. By default, everything is connected to everything; that’s what a network should do. You shouldn’t have to figure out how to route from A to B. I believe that Azure WAN will offer a superior connectivity solution, even for a single location organisation. That’s why I’ve been spending time figuring this tech out over the last few weeks.

The Office – Paint & Electrics Complete

The installation and painting of the Cloud Mechanix global HQ have completed. On a damp day, the painters arrived and started work – sealing the knots in the wood and applying the first coat. We decided to go with a different set of colours to the house; there were two reasons for the decision:

  • To separate work from home.
  • To “hide” the office in the corner and in the trees surrounding that corner.

So dark green was used on the outside walls with white for the trims. The interior walls and ceilings are also painted white to give the place a bright feel. The floors have not been painted, but more on that in a moment. The electricians arrived on the first day of painting and installed the electrical components. They didn’t have the ethernet cabling so they had to return a couple of days later to do that work. The painting took 2 full days with 2 painters – that’s without sanding and painting/staining the floor like many would opt to do.

And that brings us to the floor. We decided to install AC4 (hard wearing) laminate flooring – if you don’t know what that is, it’s artificial wood flooring that has the pattern, colour and texture of finished wood without the maintenance. We found a style we like, moor acacia, that is dark to contrast with the interior white, and has a grainy finish like oak. It will pop nicely with the white furnishings. We went to two local suppliers. Both had identical pricing on the supply and similar pricing on the installation. However, one has a month of work queued up and the other has 2 months. We tried an independent installer where we would supply the materials but the price was too high with a non-guaranteed schedule. So we’ll be waiting 1 month to get the floor installed – no, that’s not work for me – trust me, I attempted it once before in a small room that took way too long and didn’t look good after.

I installed the Wi-Fi to the fixed ethernet connection in the office using a spare Linkys mesh unit that I had. It tested well, similar to the same tests in the house, closer to the broadband modem. No; I did not opt to install RJ45 sockets all around the office. Wi-Fi will be reliable (antenna in the office), flexible, and not take away from the appearance of the interior. Not to mention that many things expect Wi-Fi connections these days anyway!

I also shopped online for furniture. I know what I want – most of it is from Ikea – but the best delivery I could get from them was 5-6 weeks!!! So the whole floor thing won’t impact us anyway. The only real decision I have not made is what to do with the office chairs. I know many of you will recommend Secret Labs but their delivery times are widely mocked.

As you can see in the photo, there are a few other works to be done:

  • The swing set is being scrapped. A much nicer wooden set is coming next week and a local handyman will take the scrap away to recycle it.
  • I had some paving stones that I had saved from the site that the office is installed on. I will be power washing them.
  • The site of the old swing set left some damage to the lawn – the anchors and foot damage. I have filled the holes and I will be sowing new grass.

Some of the garden work will happen this week because I’ve taken some time off from work. Other than that, it will be a while before there are any updates on this project. There is one other thing to share:

The above slate sign arrived in the post today. It was a present from my wife to hang on/in the office. It’s written in Norwegian (I work for Innofactor Norway) and explicitly refers to the 3 x tree stumps that had to be dug up by hand to clear space for the office.

The Office – Construction Complete

The construction of Cloud Mechanix global HQ finished yesterday afternoon. The final piece to go in place was the step up to the door. You can really see the slope in the site in the below photo.


If you step in you can see the all-wood finish, with the 1st fitting electrics, ready for the final touches.

And you can see the view from one of the locations where one of the desks will be located.

We had some paving stones from the site clearance, so they are being repurposed to cross the lawn from the house to the front door. A swing set was placed directly in-front of the office – the replacement is going to the far end of the lawn. The grass underneath and the empty anchor spots are bald so I’ll be re-sowing grass there in the coming days.

So what’s next? Paint and second-fitting electrics are next in the project plan. I will also be looking at how we can extend the house security systems to the office – a combination of a supplier-based monitored alarm system and the Ring cameras. That will allow us to insure the contents of the new office, and then … it’ll be time to move in.

Azure Virtual WAN ARM – The Resources

In this post, I will explain the types of resources used in Azure Virtual WAN and the nature of their relationships.

Note, I have not included any content on the recently announced preview of third-party NVAs. I have not seen any materials on this yet to base such a post on and, being honest, I don’t have any use-cases for third-party NVAs.

As you can see – there are quite a few resources involved … and some that you won’t see listed at all because of the “appliance-like” nature of the deployment. I have not included any detail on spokes or “branch offices”, which would require further resources. The below diagram is enough to get a hub operational and connected to on-premises locations and spoke virtual networks.

The Virtual WAN – Microsoft.Network/virtualWans

You need at least one Virtual WAN to be deployed. This is what the hub will connect to, and you can connect many hubs to a common Virtual WAN to get automated any-to-any connectivity across the Microsoft physical WAN.

Surprisingly, the resource is deployed to an Azure region and not as a global resource, such as other global resources such as Traffic Manager or Azure DNS.

The Virtual Hub – Microsoft.Network/virtualHubs

Also known as the hub, the Virtual Hub is deployed once, and once only, per Azure region where you need a hub. This hub replaces the old hub virtual network (plus gateway(s), plus firewall, plus route tables) deployment you might be used to. The hub is deployed as a hidden resource, managed through the Virtual WAN in the Azure Portal or via scripting/ARM.

The hub is associated with the Virtual WAN through a virtualWAN property that references the resource ID of the virtualWans resource.

In a previous post, I referred to a chicken & egg scenario with the virtualHubs resource. The hub has properties that point to the resource IDs of each deployed gateway:

  • vpnGateway: For site-to-site VPN.
  • expressRouteGateway: For ExpressRoute circuit connectivity.
  • p2sVpnGateway: For end-user/device tunnels.

If you choose to deploy a “Secured Virtual Hub” there will also be a property called azureFirewall that will point to the resource ID of an Azure Firewall with the AZFW_Hub SKU.

Note, the restriction of 1 hub per Azure region does introduce a bottleneck. Under the covers of the platform, there is actually a virtual network. The only clue to this network will be in the peering properties of your spoke virtual networks. A single virtual network can have, today, a maximum of 500 spokes. So that means you will have a maximum of 500 spokes per Azure region.

Routing Tables – Microsoft.Network/virtualHubs/hubRouteTables & Microsoft.Network/virtualHubs/routeTables

These are resources that are used in custom routing, a recently announced as GA feature that won’t be live until August 3rd, according to the Azure Portal. The resource control the flows of traffic in your hub and spoke architecture. They are child-resources of the virtualHubs resource so no references of hub resource IDs are required.

Azure Firewall – Microsoft.Network/azureFirewalls

This is an optional resource that is deployed when you want a “Secured Virtual Hub”. Today, this is the only way to put a firewall into the hub, although a new preview program should make it possible for third-parties to join the hub. Alternatively, you can use custom routing to force north-south and east-west traffic through an NVA that is running in a spoke, although that will double peering costs.

The Azure Firewall is deployed with the AZFW_Hub SKU. The firewall is not a hidden resource. To manage the firewall, you must use an Azure Firewall Policy (aka Azure Firewall Manager). The firewall has a property called firewallPolicy that points to the resource ID of a firewallPolicies resource.

Azure Firewall Policy – Microsoft.Network/firewallPolicies

This is a resource that allows you to manage an Azure Firewall, in this case, an AZFW_Hub SKU of Azure Firewall. Although not shown here, you can deploy a parent/child configuration of policies to manage firewall configurations and rules in a global/local way.

VPN Gateway – Microsoft.Network/vpnGateways

This is one of 3 ways (one, two or all three at once) that you can connect on-premises (branch) sites to the hub and your Azure deployment(s). This gateway provides you with site-to-site connectivity using VPN. The VPN Gateway uses a property called virtualHub to point at the resource ID of the associated hub or virtualHubs resource. This is a hidden resource.

Note that the virtualHubs resource must also point at the resource ID of the VPN gateway resource ID using a property called vpnGateway.

ExpressRoute Gateway – Microsoft.Network/expressRouteGateways

This is one of 3 ways (one, two or all three at once) that you can connect on-premises (branch) sites to the hub and your Azure deployment(s). This gateway provides you with site-to-site connectivity using ExpressRoute. The ExpressRoute Gateway uses a property called virtualHub to point at the resource ID of the associated hub or virtualHubs resource. This is a hidden resource.

Note that the virtualHubs resource must also point at the resource ID of the ExpressRoute gateway resource ID using a property called p2sGateway.

Point-to-Site Gateway – Microsoft.Network/p2sVpnGateways

This is one of 3 ways (one, two or all three at once) that you can connect on-premises (branch) sites to the hub and your Azure deployment(s). This gateway provides users/devices with connectivity using VPN tunnels. The Point-to-Site Gateway uses a property called virtualHub to point at the resource ID of the associated hub or virtualHubs resource. This is a hidden resource.

The Point-to-Site Gateway inherits a VPN configuration from a VPN configuration resource based on Microsoft.Network/vpnServerConfigurations, referring to the configuration resource by its resource ID using a property called vpnServerConfiguration.

Note that the virtualHubs resource must also point at the resource ID of the Point-to-Site gateway resource ID using a property called p2sVpnGateway.

VPN Server Configuration – Microsoft.Network/vpnServerConfigurations

This configuration for Point-to-Site VPN gateways can be seen in the Azure WAN and is intended as a shared configuration that is reusable with more than one Point-to-Site VPN Gateway. To be honest, I can see myself using it as a per-region configuration because of some values like DNS servers and RADIUS servers that will probably be placed per-region for performance and resilience reasons. This is a hidden resource.

The following resources were added on 22nd July 2020:

VPN Sites – Microsoft.Network/vpnSites

This resource has a similar purpose to a Local Network Gateway for site-to-site VPN connections; it describes the on-premises location, AKA “branch office”.  A VPN site can be associated with one or many hubs, so it is actually connected to the Virtual WAN resource ID using a property called virtualWan. This is a hidden resource.

An array property called vpnSiteLinks describes possible connections to on-premises firewall devices.

VPN Connections – Microsoft.Network/vpnGateways/vpnConnections

A VPN Connections resource associates a VPN Gateway with the on-premises location that is described by an associated VPN Site. The vpnConnections resource is a child resource of vpnGateways, so there is no actual resource; the vpnConnections resource takes its name from the parent VPN Gateway, and the resource ID is an extension of the parent VPN Gateway resource ID.

By necessity, there is some complexity with this resource type. The remoteVpnSite property links the vpnConnections resource with the resource ID of a VPN Site resource. An array property, called vpnSiteLinkConnections, is used to connect the gateway to the on-premises location using 1 or 2 connections, each linking from vpnSiteLinkConnections to the resource/property ID of 1 or 2 vpnSiteLinks properties in the VPN Site. With one site link connection, you have a single VPN tunnel to the on-premises location. With 2 link connections, the VPN Gateway will take advantage of its active/active configuration to set up resilient tunnels to the on-premises location.

Virtual Network Connections – Microsoft.Network/virtualHubs/hubVirtualNetworkConnections

The purpose of a hub is to share resources with spoke virtual networks. In the case of the Virtual Hub, those resources are gateways, and maybe a firewall in the case of Secured Virtual Hub. As with a normal VNet-based hub & spoke, VNet peering is used. However, the way that VNet peering is used changes with the Virtual Hub; the deployment is done using the hub/VirtualNetworkConnections child resource, whose parent is the Virtual Hub. Therefore, the name and resource ID are based on the name and resource ID of the Virtual Hub resource.

The deployment is rather simple; you create a Virtual Network Connection in the hub specifying the resource ID of the spoke virtual network, using a property called remoteVirtualNetwork. The underlying resource provider will initiate both sides of the peering connection on your behalf – there is no deployment required in the spoke virtual network resource. The Virtual Network Connection will reference the Hub Route Tables in the hub to configure route association and propagation.

More Resources

There are more resources that I’ve yet to document, including:

The New Office – Day 3

It’s Monday and day 3 of the construction of Cloud Mechanix Global HQ. On Friday, the carpenters finished the steel roof and installed the wall studs for the cavity wall with insulation, as you can see in the photo.

One of the options we selected in the installation was to have a “plinth” and step installed. The price for this is always estimated on the site – the site effects the levelling and height of the foundation, and the height determines the amount of wood and labour required. The quote I was given was €350. The team is planning to finish the interior and additional wood work today. Next up will be the painters and electrical second fitting. We also added the options for paint and painting in the purchase. I can paint, but I suck at edges and I’m slow; I’d rather let a professional do it plus that means it’s time I can spend doing other things.

Thoughts are now turning to the interior, which will be painted white. Floorboards are being installed and also being painted white. But I am considering installing additional flooring to protect the structural wood. If I do, it’ll be something dark, or even grey, to contrast somewhat with the white walls and ceiling. I already have a white L-shaped Ikea “Bekant” desk. I think I will stick with the Ikea desks – I do like them. The plan is that the L-shaped desks will be placed behind the windows at the front, giving us a nice view up the garden while working. We’ll see where the budget is in a week or so, but I’m partial towards the version of the Bekant that includes powered legs that can raise/lower the desk at the push of a button.

I have an office chair already with a white frame and black cushions. But it was a bad purchase – the piston lasted only a few months and it makes a cracking noise when I adjust my position – that was why I replaced the previous seat! I can see myself buying some Secret Lab chairs so comments on that are welcome – from people who have owned one for more than 1 year.

I have a white 2×4 Ikea “Kallax” unit in the small office in the house at the moment. On the plus side, it’s great to display certain things. On the negative side, the close options are too small for the stuff you want to hide – I have 2 large plastic storage boxes hidden in the built-in wardrobe in the office at the moment, filled with things like spare peripherals, cables, battery packs, and so on. I think I want something that is closed storage at the bottom and open shelf storage at the top. I will keep looking.

My desk is quite full at the moment: 3 x USB drives (that I might replace with a NAS), a laptop, an Intel NUC, a KVM switch, a Surface dock, an Epson eco-tank printer, 2 x 27″ monitors, a mic, and so on. Some of the stuff is stuff that you need around but don’t need beside you all the time. Because we have a nearly 6 x 4m space, we have room to add more desk space. So I am thinking of putting in 1, or even 2, additional straight desks at a later time. Things like the printer and networking gear can be relocated to there.

Finally, there are times where you want to work, but want to relax. Maybe you need to read something in a more relaxed position, or just take a break from the screen. I’m thinking that we’ll put in a black leather couch in middle of the office – I’m not sure where yet because I might also move the 43″ TV from the house office into there – nice for watching conferences, etc. The second-hand market sometimes is great for that kind of thing – my mother-in-law is quite skilled at buying/selling on those kinds of websites so we might have to recruit her assistance. Depending on how that goes and how much space there is, we might add a small coffee table too.

The New Office – Construction Has Begun

It is day 2 of the construction of our new office. The carpenters arrived exactly at 9 am yesterday morning, bang-on time, to begin the assembly and construction of our new garden office – the global headquarters of Cloud Mechanix.

My only concern at the start was whether the site would be level enough for the build. The team warned us early on that the slope would make one side look higher up when it was level but we knew that and were OK with it. They quickly assembled the wooden frame that would eventually site on concrete blocks.

Once the 6x4m frame was built I was called out to pick the final position – allowing just over half a metre from the concrete garden wall so I can paint the outside to maintain the integrity of the wooden construction. Not long after the wooden frame was in position. Around lunchtime, the planks for the outside wall were being hammered into place. I’d say it took the 3 of them 45 minutes to get all the planks in place. Then the ceiling was put on.

The electricians were called in and asked to confirm how we wanted to do things after they surveyed the site. The one thing I was worried about was how the armoured electrical cable would run from the office to the fuse box at the front of the house – we need to power at least 2 computers + equipment and a 1.5 KW heater (please save comments on €13,000 hearing/aircon systems – the office is costing that much!). The ethernet line and electrical cable are entering on the back-right and will run along the garden wall under the capping (which you cannot see in the photos here). Then they will run underground to the front of the house. The ethernet line will enter the house at the same location as the phone/Internet and be just a meter from the broadband router. The electrical cable will cross the front of the house hidden in white trunking (maybe 2 cm wide) attached under the white soffit of the house – nicely camouflaged. The electrical cable will enter above the front door (still camouflaged) and then be under 1 metre from the fusebox – trunking from the front door, along an interior white doorframe – still camouflaged.

The electricians did the first fitting: 5 x dual power sockets, light switch, and 6 x dropdown lights. When leaving, they let me know that they would be back once the interior was finished by the carpenters to do the second fittings – sockets, lights, and the aforementioned cable installations.

The carpenters carried on, adding insulation and sheeting to the roof before the end of the day.

As you can see, there are two big windows, right where the front desks will be – one for me and one for my wife. Cloud Mechanix will have a great view overlooking about 40m of the garden!

Day two (today) began with drizzle. The team (down to two) were bang-on time again. They told me that they expect to require 3 days for the complete assembly and finishing of the wooden structure. Today, the plan was to get the doors and windows done, the studs for the wall insulation & interior installed, and the roof started. As you can see, the doors and windows were in place at lunchtime.

Exterior trims have been added, the slats for the roof (steel roof) are in place, and I can hear steel being cut as I type – they are installing the steel sheet roof that has a black tile effect. Once that is done, the slow bit (as they referred to it) will begin: the interior finishes, the lower trim and the step.

Thanks to Masaaki Komori for sharing their work on Unsplash.

Azure Virtual WAN ARM – The Chicken & Egg Gateway ID Discombobulation

This post will explain how to deal with the gateway ID properties in the Azure Microsoft.Network/virtualhubs resource when using ARM templates.

Background

The Azure WAN Hub is capable of having 3 gateway sub-resources:

  • Point-to-site VPN: Microsoft.Network/p2sVpnGateways
  • VPN (site-to-site): Microsoft.Network/vpnGateways
  • ExpressRoute: Microsoft.Network/expressRouteGateways, which does not support diagnostic settings in the 2020-04-01 API

As you would expect, when you create these resources, you have to supply them with the resource ID of the Microsoft.Network/virtualhubs resource:

What is a surprise is what happens in the Microsoft.Network/virtualhubs resource. After a gateway is associated, a property (type object, presumably for future-proofing) for the associated gateway type is added to the hub:

The surprising thing is what happens.

The Problem

There are 3 possible states in the hub when it comes to each gateway:

  1. The hub exists without a gateway: The above hub properties are not required.
  2. The gateways are being added: The above hub properties cannot be added because the gateway resource ID points to a resource that does not exist yet – the hub must exist and be configured before the gateway(s).
  3. The gateways exist: Any re-run of the ARM template (which might be common to update the hub route tables or configuration via DevOps) must include the above gateway properties in the hub resource with the correct resource IDs for the gateways.

And steps 2 and 3 are where the chicken and egg are in an ARM template. You must supply the gateway resource ID in the hub for all updates to the hub after a gateway is deployed, and you must not include the gateway resource ID in the hub when deploying the gateway. This would be easy to deal with if ARM would (finally) give us a “ifexists()” function but there is no sign of that. So we need a hack solution.

The Hack Solution

This one comes from the Well-Architected Framework/Cloud Adoption Framework, Enterprise-Scale Architecture. This way-too-complicated beastie shows how Microsoft’s people are dealing with the issue. The JSON for the Microsoft.Network/virtualhubs template contains these properties:

The key for dealing with vpnGateway is the vHUB parameter, an object that contains a value called vpnGateway.

When they first run the deployment, the value of vHUB.vpngateway is set to {} or null in the parameters file, stored in GitHub. That means that when the hub is first run (and there is no VPN gateway), the if statement in the above snippet will pass json(‘null’) to the vpnGateway property. That is acceptable to the resource provider and the hub will deploy cleanly. Later on in the deployment, the VPN gateway will be created.

If you were to just re-run the hub template now, you will get an error about not being allowed to change the vpnGateway property in the hub resource. Behind the scenes it has been updated by the VPN gateway deployment. Every execution of the hub template must now include the resource ID of the VPN Gateway – that sucks, right? Now the hack really kicks in.

After the first deployment of the hub (and the VPN Gateway), you must open the resource group in the Azure Portal, enable viewing hidden items, open the VPN Gateway resource, go to properties, and document the resource ID.

Now, you need to open the parameters file for the hub. Edit the vHUB.vpnGateway property and set it to:

Now you can cleanly re-run the hub template.

How Should It Work?

The best solution would be if the gateway ID properties were just documentation for Azure, properties that we humans cannot edit. But I suspect that the ability to configure these settings might have something to do with the newly announced NVA-in-hub preview. Otherwise, ARM needs to finally give us an ifexists() function – vote here now if you agree.

Azure Virtual WAN ARM – Secured Virtual Hub Azure Firewall

I have spent quite a few hours figuring out how to deploy Azure’s new Secured Virtual Hub, an extension of Azure Virtual WAN, deployed using ARM templates (JSON). A lot of the bits are either not documented or incorrectly documented. One of the frustrating bits to deploy was the Azure Firewall resource – and the online examples did not help.

The issue was that the 2 sources I could find did not include public IP addresses on the firewall:

  • The quick start for Secured Virtual Hub on docs.microsoft.com
  • The new Enterprise-Scale “well-architected” Framework, found in Cloud Adoption Framework

Digging to solve that uncovered:

  • The examples used quite an old API version, 2019-08-01, to deploy the Microsoft.Network/azureFirewalls resource.
  • There was no example of how to add a public IP address to the firewall in Secured Virtual Hub because it was not possible with that API – SVH is quite different from a VNet deployment because you do have direct access to the underlying hub virtual network.
  • Being an old API, we lose features such as SNAT for non-RFC1918 addresses (important in universities and public sector) and the newer custom & proxy DNS features.

In my digging, I did uncover that the ARM reference for the Azure Firewall was incorrect, but I did uncover a new, barely-documented property called hubIPAddresses; I knew this property was the key to solving the public IP address issue. So I thought about what was going on and how I was going to solve it.

I ended up doing what I would normally do if I did not have a quick start template to start with:

  1. Deploy the resource(s) by hand in the Azure Portal
  2. Observe the options – there was a slide control for the quantity of firewall public IP addresses
  3. Export the resulting template

And … there was the solution:

  1. There is a new, undocumented API version for the Azure Firewall resource: 2020-05-01
  2. There is a new object property called hubIPAddresses that contains an object sub-property called publicIps. You can set a string value called count to control how many public IP addresses that Azure will assign (on your behalf) to the firewall – you do not need to create the public IP address resources.

Sorted!

The Office Installation Is About To Start

It’s a while (about 6 weeks) since I wrote that I am getting a new office built in the garden. Today seems like a good time to post an update.

There were 3 birch trees on the site where the office is being installed. I had the trees cut down (we have already replaced them with 4 apple trees and 2 more will be added next year) and that left the stumps. If I had been clever, I would have hired a mini-digger and dug around them. Maybe I would have also hired a tree grinder to chop up the revealed stumps. But no – instead we decided to dig them out by hand. Birch trees have interesting roots. The root ball is about 60-80 CM deep, and roots, sometimes 3x wider (or more) than the trunk, come out and up towards the surface. Those roots break out into a fine mesh that spreads out around the site, holding the soil firmly together, almost like concrete. I had a pick-axe, an axe, a shovel, a trowel, and my bare hands. The pick-axe was literally bouncing off the ground. The roots were like a phone book, and a sharp axe wasn’t cutting; it was breaking a layer or two and bounding too. I’ll keep this short – the final stump took over 8 hours to dig out by myself, and ended with me using my hands to scoop out dirt from under the root ball and roll the stump until the tap-root snapped.

Tip: hire the machinery, and don’t do it by hand.

The next thing to deal with was the base. Once the trees were removed, we realised that the site had a slant that we had never noticed before – about 27cm from left to right over 6m. I thought that would require a concrete base. So I put out some feelers to find a builder that would install the base. I got 4 names. Only 3 returned my contact attempts. Of the 3, only 2 bothered to show up – one of those was 2 days late! And of the 2, neither either bothered to give me a quote. I guess that builders are busy enough, even with all the COVID disruption that’s been going on. I called the cabin builders to get their opinion – and they said that we didn’t need a base. Their method of foundation installation would handle the minor slope. Perfect!

The electrician was the next thing. After my last post, I had lots of people telling me to go spend another 13,000-20,000 Euros on different kinds of heating systems – me arse! We didn’t even spend that on the house! I got an electrician quote – a guy who came recommended from the cabin builders. I’d tried reaching out to one other electrician but he was as useless at responding as the builders. The recommended guy is booked and will be doing the first (armored cable from the fusebox) and second fittings, as well as an Ethernet wire installation from our broadband router in the house to the office.

The last piece of the installation will be painting – that’s paid for with the installation and will be done afterward.

So when will all this happen? A lorry dropped off an “Ikea-like” load in our driveway a couple of hours ago. An installation team is due tomorrow at 9am to start the assembly. Hopefully we won’t have to wait too long for the painters and then we can get the alarm/security stuff installed, a new floor surface installed, and purchase some new office furniture.

Azure Virtual WAN Introducing A New Kind Of Route Table

In this post, I will quickly introduce you to a new kind of Route Table in Microsoft Azure that has been recently introduced by Azure Virtual WAN – and hence is included in the newly generally available Secured Virtual Hub.

The Old “Subnet” Route Table

This Route Table, which I will call “Subnet Route Table” (derived from the ARM name) is a simple resource that we associate with a subnet. It contains User-Defined Routes that force traffic to flow in desirable directions, typically when we use some kind of firewall appliance (Azure Firewall or third-party) or a third-party routing appliance. route The design is simple enough:

  • Name: A user-friendly name
  • Prefix: The CIDR you want to get to
  • Next Hop Type: What kind of “router” is the next hop, e.g. Virtual Network, Internet, or Virtual Appliance
  • Next Hop IP Address: Used when Next Hop Type is Virtual Appliance (any firewall or third-party router)

Azure Virtual WAN Hub

Microsoft introduced Azure Virtual WAN quite a while ago (by Cloud standards), but few still have heard of it, possibly because of how it was originally marketed as an SD-WAN solution compatible originally with just a few on-prem SD-WAN vendors (now a much bigger list). Today it supports IKEv1 and IKEv2 site-to-site VPN, point-to-site VPN, and ExpressRoute Standard (and higher). You might already be familiar with setting up a hub in a hub-and-spoke: you have to create the virtual network, the Route Table for inbound traffic, the firewall, etc. Azure Virtual WAN converts the hub into an appliance-like experience surfacing just two resources: the Virtual WAN (typically 1 global resource per organisation) and the hub (one per Azure region). Peering, routing, connectivity are all simplified.

A more recent change has been the Secured Virtual Hub, where Azure Firewall is a part of the Virtual WAN Hub; this was announced at Ignite and has just gone GA. Choosing the Secured Virtual Hub option adds security to the Virtual WAN Hub. Don’t worry, though, if you prefer a third-party firewall; the new routing model in Azure Virtual WAN Hub allows you to deploy your firewall into a dedicated spoke virtual network and route your isolated traffic through there.

The New Route Tables

There are two new kinds of route table added by the Virtual WAN Hub, or Virtual Hub, both of which are created in the Virtual Hub as sub-resources.

  • Virtual Wan Hub Route Table
  • Virtual WAN Route Table

Virtual WAN Hub Route Table

The Virtual Hub Hub Route Table affects traffic from the Virtual Hub to other locations.  A possible scenario is when you want to route traffic to a CIDR block of virtual network(s) through a third-party firewall (network virtual appliance/NVA):

AzureVirtualWanHubHubRouteTable

The routing rule setup here is similar to the Subnet Route Table, specifying where you want to get to (CIDR, resource ID, or service), the next hop, and a next hop IP address.

Virtual WAN Route Table

The Virtual WAN Route Table is created as a sub resource of the Virtual Hub but it has a different purpose. The Virtual Hub is assigned to connections and affects routing from the associated branch offices or virtual networks. Whoa, Finn! There is a lot of terminology in that sentence!

A connection is just that; it is a connection between the hub and another network. Each spoke connected directly to the hub has a connection to the hub – a Virtual WAN Route Table can be associated with each connection. A Virtual WAN Route Table can be associated with 1 virtual network connection, a subset of them, or all of them.

The term “branch offices” refers to sites connected by ExpressRoute, site-to-site VPN, or point-to-site VPN. Those sites also have connections that a Virtual WAN Route Table can be associated with.

This is a much more interesting form of route table. I haven’t had time to fully get under the covers here, but comparing ARM to the UI reveals two methodologies. The Azure Portal reveals one way of visualising routing that I must admit that I find difficult to scale in my mind. The ARM resource looks much more familiar to me, but until I get into a lab and fully test (which I hope I will find some hours to do soon), I cannot completely document.

Here are the basics of what I have gleaned from the documentation, which covers the Azure Portal method:

The linked documentation is heavy reading. I’m one of those people that needs to play with this stuff before writing too much in detail – I never trust the docs and, to be honest, this content is complicated, as you can see above.