This post will explain how routing works in Microsoft Azure, and how to troubleshoot your routing issues with Route Tables, BGP, and User-Defined Routes in your virtual network (VNet) subnets and virtual (firewall) appliances\/Azure Firewall.<\/p>\n
Right now, you need to forget VLANs, and how routers, bridges, routing switches, and all that crap works in the physical network. Some theory is good, but the practice \u2026 that dies here.<\/p>\n
Azure networking is software-defined (VXLAN). When a VM sends a packet out to the network, the Azure Fabric takes over as soon as the packet hits the virtual NIC. That same concept extends to any virtual network-capable Azure service. From your point of view, a memory copy happens from source NIC to destination NIC. Yes; under the covers there is an Azure backbone with a \u201cmore physical\u201d implementation but that is irrelevant because you have no influence over it.<\/p>\n
So always keep this in mind: network transport in Azure is basically a memory copy. We can, however, influence the routing of that memory copy by adding hops to it.<\/p>\n
When you create a VNet, it will have 1 or more subnets. By default, each subnet will have system routes. The first ones are simple, and I\u2019ll make it even more simple:<\/p>\n
By the way, the only way to see system routes is to open a NIC in the subnet, and click Effective Routes under Support & Troubleshooting. I have asked that this is revealed in a subnet \u2013 not all VNet-connected services have NICs!<\/p>\n
And also, by the way, you cannot ping the subnet default gateway because it is not an appliance; it is a software-defined function that is there to keep the guest OS sane \u2026 and probably for us too \ud83d\ude0a<\/p>\n
When you peer a VNet with another VNet, you do a few things, including:<\/p>\n
The result is that VMs in VNet1 will send packets directly to VMs in VNet2 as if they were in the same VNet.<\/p>\n
When you create a VNet gateway (let\u2019s leave BGP for later) and create a load network connection, you create another (set of) system routes for the virtual network gateway. The local address space(s) will be added as destinations that are tunnelled via the gateway. The result is that packets to\/from the on-prem network will route directly through the gateway \u2026 even across a peered connection if you have set up the hub\/spoke peering connections correctly.<\/p>\n
Let\u2019s add BGP to the mix. If I enable ExpressRoute or a BGP-VPN, then my on-prem network will advertise routes to my gateway. These routes will be added to my existing subnets in the gateway\u2019s VNet. The result is that the VNet is told to route to those advertised destinations via the gateway (VPN or ExpressRoute).<\/p>\n
If I have peered the gateway\u2019s VNet with other VNets, the default behaviour is that the BGP routes will propagate out. That means that the peered VNets learn about the on-premises destinations that have been advertised to the gateway, and thus know to route to those destinations via the gateway.<\/p>\n
And let\u2019s stop there for a moment.<\/p>\n
We now have 2 kinds of route in play \u2013 there will be a third. Let\u2019s say there is a system route for 172.16.0.0\/16 that routes to virtual network. In other words, just \u201cfind the destination in this VNet\u201d. Now, let\u2019s say BGP advertises a route from on-premises through the gateway that is also for 172.16.0.0\/16.<\/p>\n
We have two routes for the 172.16.0.0\/16 destination:<\/p>\n
Azure looks at routes that clash like above and deactivates one of them. Azure always ranks BGP above System. So, in our case, the System route for 172.16.0.0\/16 will be deactivated and no longer used. The BGP route for 172.16.0.0\/16 via the VNet gateway will remain active and will be used.<\/p>\n
Try saying that word 5 times in a row after 5 drinks!<\/p>\n
The most specific route will be chosen. In other words, the route with the best match for your destination is selected by the Azure fabric. Let\u2019s say that I have two active routes:<\/p>\n
Now, let\u2019s say that I want to send a packet to 172.16.1.4. Which route will be chosen? Route A is a 16 bit match (172.16.*.*). Route B is a 24 bit match (172.16.1.*). Route B is a closer match so it is chosen.<\/p>\n
Now add a scenario where you want to send a packet to 172.16.2.4. At this point, the only match is Route A. Route B is not a match at all.<\/p>\n
This helps explain an interesting thing that can happen in Azure routing. If you create a generic rule for the 0.0.0.0\/0 destination it will only impact routing to destinations outside<\/em> of the virtual network \u2013 assuming you are using the private address spaces in your VNet. The subnets have system routes for the 3 private address spaces which will be more specific than 0.0.0.0:<\/p>\n If your VNet address space is 10.1.0.0\/16 and you are trying to send a packet from subnet 1 (10.1.1.0\/24) to subnet 2 (10.1.2.0\/24), then the generic Route D will always be less specific than the system route, Route C.<\/p>\n A route table resource allows us to manage the routing of a subnet. Good practice is that if you need to manage routing then:<\/p>\n The first thing to know about route tables is that you can control BGP propagation with them. This is especially useful when:<\/p>\n The default is that BGP propagation is allowed over a peering connection to the spoke. In the route table (Settings > Configuration) you can disable this propagation so the BGP routes are never copied from the hub network (with the VNet gateway) to the peered spoke VNet\u2019s subnets.<\/p>\n The second thing about route tables is that they allow us to create user-defined routes (UDRs).<\/p>\n You can control the flow of packets using user-defined routes. Note that UDRs outrank BGP routes and System Routes:<\/p>\n If I have a system or BGO route to get to 192.168.1.0\/24 via some unwanted path, I can add a UDR to 192.168.1.0\/24 via the desired path. If the two routes are identical destination matches, then my UDR will be active and the BGP\/system route will be deactivated.<\/p>\n The traditional tool you might have used is TRACERT. I\u2019m sorry, it has some use, but it\u2019s really not much more than PING. In the software defined world, the default gateway isn\u2019t a device with a hop, the peering connection doesn\u2019t have a hop, and TRACERT is not as useful as it would have been on-premises.<\/p>\n The first thing you need is the above knowledge. That really helps with everything else.<\/p>\n Next, make sure your NSGs aren\u2019t the problem, not your routing!<\/p>\n Next is the NIC, if you are dealing with virtual machines. Go to Effective Routes and look at what is listed, what is active and what is not.<\/p>\n Network Watcher has a couple of tools you should also look at:<\/p>\n The tools won\u2019t tell you why a routing plan failed, but with the above information, you can troubleshoot a (desired) network path.<\/p>\n","protected":false},"excerpt":{"rendered":" This post will explain how routing works in Microsoft Azure, and how to troubleshoot your routing issues with Route Tables, BGP, and User-Defined Routes in your virtual network (VNet) subnets and virtual (firewall) appliances\/Azure Firewall. Software-Defined Networking Right now, you need to forget VLANs, and how routers, bridges, routing switches, and all that crap works … Continue reading “How to Troubleshoot Azure Routing?”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":18983,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_uf_show_specific_survey":0,"_uf_disable_surveys":false,"footnotes":""},"categories":[5],"tags":[330,170,328,242,320,307,253,305,325,324,319,329,327,326,289,281],"class_list":["post-21480","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-azure","tag-appliance","tag-azure","tag-bgp","tag-firewall","tag-network-security-group","tag-network-virtual-appliance","tag-nsg","tag-nva","tag-route-table","tag-routing","tag-subnet","tag-system-route","tag-udr","tag-user-defined-routing","tag-virtual-network","tag-vnet"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n
Route Tables<\/h2>\n
\n
\n
User-Defined Routes<\/h2>\n
\n
Troubleshooting Tools<\/h2>\n
\n