In this post, I will explain how things such as frontend configurations, listeners, HTTP settings, probes, backend pools, and rules work together to enable service publication in the Azure Web Application Gateway (WAG)/Web Application Firewall (WAF).
Introduction
The WAF/WAG is a scary beast at first. When you open one up there are just so many settings to be tweaked. If you are publishing just a simple test HTTP server, it’s easy: you populate the default backend pool and things just start to work. But if you want HTTPS, or to service many pools/sites, then things get complicated. And frustratingly slow 🙂 – Things have improved in v1 and v2 is significantly faster to configure, although it has architectural limitations (force public IP address and lack of support for route tables) that prevent me from using v2 in my large network deployments. Hopefully, the above map and following text will simplify things by explaining what all the pieces do and how they work together.
The below is not feature complete, and things will change in the future. But for 99% of you, this should (hopefully) be helpful.
Backend Pool
The backend pool describes a set of machines/services that will work together. The members of a backend pool must be all of the same type from one of these types:
- IP address/hostname: a common choice in large Azure deployments – you can span peering connections to other VNets
- Virtual machine: Select a machine from the same VNet as the WAG/WAF
- VMSS: Virtual machine scale sets in the same VNet as the WAG/WAF
- App Services: In the same subscription as the WAG/WAF
From here on out, I’ll be using the term “web server” to describe the above.
Note that this are the machines that host your website/service. They will all run the same website/service. And you can configure an optional custom probe to test the availability of the service on these machines.
(Optional) Health Probe
You can create a HTTP/HTTPS probe to do deeper probe tests of a service running on a backend pool. The probe is configured for HTTP or HTTPS and tests a hostname on the web server. You specify a path on the website, a frequency, timeout and allowed number of retries before designating a web site on a web server as being unhealthy and no longer a candidate for load balancing.
HTTP Setting
The HTTP setting configures how the WAG/WAF will talk to the members of the backend pool. It does not configure how clients talk to the site (Listener). So anything you see below here is for configuring WAG/WAF to web server communications (see HTTPS).
- Control cookie-based affinity for load balancing
- Configure connection draining when a machine is removed from a backend pool
- Specify if this is for a HTTP or a HTTPS connection to the webserver. This is for end-to-end encryption.
- For HTTPS, you will upload a certificate that will match the web servers’ certificate.
- The port that the web server is listening on.
- Override the path
- Override the hostname
- Use a custom probe
Remember that the above HTTPS setting is not required for website to be published as SSL. It is only required to ensure that encryption continues from the WAG/WAF to the web servers.
Frontend IP Configuration
A WAG/WAF can have public or private frontend IP addresses – the variation depends on if you are using V1 (you have a choice on the mix) or V2 (you must use public and private). The public front end is a single public IP address used for publishing services publicly. The private frontend is a single virtual network address used for internal service publication, requiring virtual network connectivity (virtual network, VPN, ExpressRoute, etc).
The DNS records for your sites will point at the frontend IP address of the WAG/WAF. You can use third-party or Azure DNS – Azure DNS has the benefit of being hosted in every Azure region and in edge sites around the world so it is faster to resolve names than some DNS hoster with 3 servers in a single continent.
A single frontend can be shared by many sites. http://www.aidanfinn.com, http://www.cloudmechanix.com and http://www.joeeleway.com can all point to the same IP address. The hostname configuration that you have in the Listener will determine what happens to the incoming traffic afterwards.
Listener
A Listener is configured to listen for traffic destined to a particular hostname and port number and forward it, eventually, to the correct backend pool. There are two kinds of listener:
- Basic: For very simple configurations where a site has exclusive ownership over a port number on one of the frontends. Typically this is for point solutions where a WAG/WAF is dedicated to a service.
- Multi-Site: A listener shares a frontend configuration with other listeners, and is looking for traffic destined to a specific hostname/port/protocol.
Note that the Listner is where you place the certificate to secure client > WAG/WAF communications. This is known as SSL offloading. If you enable HTTPS you will place the “site certificate” on the WAG/WAF via the Listener. You can optionally re-encrypt traffic to the webserver from the WAG/WAF using the previously discussed HTTP Setting. WAGv2/WAFv2 have a no-support preview to use certs that are securely stored in Key Vault.
The configuration of a basic listener is:
- Frontend
- Frontend port
- HTTP or HTTPS protocol
- The certificate for securing client > WAG/WAF traffic
- Optional custom error pages
The multi-site listener is adds an extra configuration: hostname. This is because now the listener is sharing the frontend and is only catching traffic for its website. So if I want 3 websites on my WAG/WAF sharing a frontend, I will have 3 x HTTPS listeners and maybe 3 x HTTP listeners.
Rules
A rule glues together the configuration. A basic rule is pretty easy:
- Traffic comes into a Listener
- The HTTP Setting determines how to forward that traffic to the backend pool
- The Backend Pool lists the web servers that host the site
A path-based rule allows you to extend your site across many backend pools. You might have a set of content for /media on pool1. Therefore all http://www.aidanfinn.com/media content is pulled from that pool1. All video content might be on http://www.aidanfinn.com/video, so you’ll redirect /video to pool2. And so on. And you can have individual HTTP settings for each redirection.
My Tips
- There’s nothing like actually setting this up at scale to try this out. You will need a few DNS names to be able to work with.
- Remember to enable the protection mode of WAF. I have audited deployments and found situations where people thought they had Layer-7 security but only had the default “alert-only” configuration of WAFv1.
- In large environments, don’t forget to ensure that the NSGs protecting any webservers allow traffic in from the WAG/WAF’s subnet into the web servers on the port(s) specified in the HTTP Setting(s). Also ensure that any guest OS firewall is similarly configured.
- Possibly the biggest issue you will have is with devs not assigning hostnames to websites in their webservers. If you’re using shared WAGs/WAFs you must use multi-site listeners and the websites should be configured with the hostname.
- And the biggest tip I can give is to work out a naming standard for each of the above components so you know what piece is associated with what site. I can’t share what we’re using at work, but we have some big configurations and they are very easy to troubleshoot because of how we have named things.
Thanks for the article. Can you also advise whether there is a need of additional load balancer infront of WAF. In one of your other articles:-
https://aidanfinn.com/?p=21474
you have mentioned the need for Load balancing of the WAG/WAF instances.
In what scenarios we will use additional load balancer.
None. The load balancer in that article resides under the covers. I talk about it because you need to allow probe traffic from it. In a private deployment, it also consumes an IP address from your subnet.
hi good article. how do you deal with redirects? I’m having issues getting data out on a specific path to a 3rd party that’s not part of a backend pool. The option is there but it doesn’t seem to work.
eg site.mydomain.com – API endpoint
site.mydomani.com/ocr – sends a stream to a service to OCR
Thanks
Can you explain a bit more? Like what’s where?
Do you know whether WAF was actually fixed in v2? WAF in v1 made our websites unusable, they were that slow. No help at all from Microsoft so we had to disable it.
You’ll have to be more specific on the root cause. I made the decision to switch to v2 last December for the following reasons:
– Enhanced policy management via WAF Policy resource
– Cert storage in Key Vault which is more secure and easier to manage via ARM than the old method
– Much faster manual operations
– It’s like other things in Azure – this is where future improvement will be
This is the best explanation of this subject I have seen!
Thank you for that!
I have still one question, is it possible to test configuration without dns name? For example using hosts file somehow?
Yes. Or do a single Basic listener and connect to the public IP of the gateway/WAF.
I see issues with the app gateway when I use custom ports for listeners. Have you faced this?
My app gateway works like a charm when its listens on 443 and route to custom port on the backend pool.
But when I change 443 to custom port and put that in the URL, it stops working.
To be clear, when I say it doesn’t work, it is not a 404 error. It seems like doesn’t process it.
https://mydomain.com/path1/path2/path3 -> works
https://mydomain.com:7777/path1/path2/path3 -> doesn’t work
Am I missing anything? Would appreciate any suggestions on this.
Hi Aidan,
When configuring multiple Application Gateways for different teams/customers, is there any downside to putting all the App Gw’s on the same VNET/Subnet. Say configure a /26 and add all the Application Gateways to it instead of creating a small network for each which uses up a lot of IP’s with losing 5 to Azure for each network.
It depends. A single VNet usually means a small set of security experts running your WAFs – that’s not bad! But on the downside, as you’ve noted, you are limiting scale-out and quantity of WAGs by the quantity of available IPs. Don’t forget that you can use multi-site listeners to host multiple apps – the rare monster app will require all the available CPU.
Is there a typo in the diagram where step 1 resolves the URL to 129.212.1.1 but then step 2 requests from public IP 129.228.1.1 ?
Hi there, I have downgraded from WAF v2 to WAF v1 as I have a small web app and the costs was too high for such small app. However I’m trying to find out how to define a WAF policy rule to whitelist a number of IP addresses for my CDN to work properly. All documentation for waf custom rules seems to point to WAF v2. Any help would much appreciated. Thanks a lot.
I’m afraid you need WAFv2 or go with an NVA if you want to do it in the WAF ruleset. Otherwise, whitelist IPs using NSGs.
I am moving from an F5 LTM to using application gateway and one of the things I have not yet figured out is how to see whether or not connections have drained from a backend member. I know that you can set connection draining but how can you tell that an individual pool member no longer has connections to it, as in even persistent sessions are all dropped? Also what is the recommended method for removing a pool member to perform maintenance/ code updates on the backend? I have seen some people use a health monitor looking for a particular file, then they rename the file so that the health check fails. This seems to be recognized quickly with aggressive polling which is pretty good, but how do I know that the existing connections to the changed node are completely drained? All metrics appear to be “sum”. Thanks.
Excellent Article, Thanks very much!
Could you please help me to clarify my doubt regarding basic listener. (WAF v2 SKU)
Would it be possible to create multiple “Basic” Listeners with the same port number (example: port 80)
Below mentioned, is the error that I’m getting when I tried to create another listener with the same port number
“This listener cannot use the same frontend port as an existing listener.”
I know listener is a logical entity/component in AAG, but I am confused and not able to understand the reason why I cannot create multiple “Basic” Listeners with the same port number. (Also, couple of articles (images) in the internet shows they have multiple “Basic” listeners with the same port number.
Thanks in advance!