Private Connections to Azure PaaS Services

In this post, I’d like to explain a few options you have to get secure/private connections to Azure’s platform-as-a-service offerings.

Express Route – Microsoft Peering

 

ExpressRoute comes in a few forms, but at a basic level, it’s a “WAN” connection to Azure virtual networks via one or more virtual network gateways; Customers this private peering to connect on-premises networks to Azure virtual networks over an SLA-protected private circuit. However, there is another form of peering that you can do over an ExpressRoute circuit called Microsoft peering. This is where you can use your private circuit to connect to Microsoft cloud services that are normally connected to over the public Internet. What you get:

  • Private access to PaaS services from your on-premises networks.
  • Access to an entire service, such as Azure SQL.
  • A wide array of Azure and non-Azure Microsoft cloud services.

FYI, Office 365 is often mentioned here. In theory, you can access Office 365 over Microsoft peering/ExpressRoute. However, the Office 365 group must first grant you permission to do this – the last I checked, you had to have legal proof of a regulatory need for private access to Cloud services. 

Service Endpoint

Imagine that you are running some resources in Azure, such as virtual machines or App Service Environment (ASE); these are virtual network integrated services. Now consider that these services might need to connect to other services such as storage accounts, Azure SQL, or others. Normally, when a VNet connected resource is communicating with, say, Azure SQL, the packets will be routed to “Internet” via the 0.0.0.0/0 default route for the subnet – “Internet” is everywhere outside the virtual network, not necessarily The Internet. The flow will hit the “public” Azure backbone and route to the Azure SQL compute cluster. There are two things about that flow:

  • It is indirect and introduces latency.
  • It traverses a shared network space.
  • A growing number of Azure-only services that support service endpoints.

A growing number of services, including storage accounts, Azure SQL, Cosmos DB, and Key Vault, all have services endpoints available to them. You can enable a service endpoint anywhere in the route from the VM (or whatever) to “Internet” and the packets will “drop” through the service endpoint to the required Azure service – make sure that any firewall in the service accepts packets from the private subnet IP address of the source (VM or whatever). Now you have a more direct and more private connection to the platform service in Azure from your VNet. What you get:

  • Private access to PaaS services from your Azure virtual networks.
  • Access to an entire service, such as Azure SQL, but you can limit this to a region.

Service Endpoint Trick #1

Did you notice in the previous section on service endpoints that I said:

You can enable a service endpoint anywhere in the route from the VM (or whatever) to “Internet”

Imagine you have a complex network and not everyone enables service endpoints the way that they should. But you manage the firewall, the public IPs, and the routing. Well, my friend, you can force traffic to support Azure platform services via service endpoints. If you have a firewall, then your routes to “Internet” should direct outbound traffic through the firewall. In the firewall (frontend) subnet, you can enable all the Azure service endpoints. Now when packets egress the firewall, they will “drop” through the service endpoints and to the desired Azure platform service, without ever reaching “Internet”.

Service Endpoint Trick #2

You might know that I like Azure Firewall. Here’s a trick that the Azure networking teams shared with me – it’s similar to the above one but is for on-premises clients trying to access Azure platform services.

You’ve got a VPN connection to a complex virtual network architecture in Azure. And at the frontend of this architecture is Azure Firewall, sitting in the AzureFirewallSubnet; in this subnet you enabled all the available service endpoints. Let’s say that someone wants to connect to Azure SQL using Power BI on their on-premises desktop. Normally that traffic will go over the Internet. What you can do is configure name resolution on your network (or PC) for the database to point at the private IP address of the Azure Firewall. Now Power BI will forward traffic to Azure Firewall, which will relay you to Azure SQL via the service endpoint. What you get:

  • Private access to PaaS services from your on-premises or Azure networks.
  • Access to individual instances of a service, such as an Azure SQL server
  • A growing number of Azure-only services that support service endpoints.

Private Link

In this post, I’m focusing on only one of the 3 current scenarios for Private Link, which is currently in unsupported preview in limited US regions only, for limited platform services – in other words, it’s early days.

This approach aims to give a similar solution to the above “Service Endpoint Trick #2” without the use of trickery. You can connect an instance of an Azure platform service to a virtual network using Private Link. That instance will now have a private IP address on the VNet subnet, making it fully routable on your virtual network. The private link gets a globally unique record in the Microsoft-managed privatelink.database.windows.net DNS zone. For example, your Azure SQL Server would now be resolvable to the private IP address of the private link as yourazuresqlsvr.privatelink.database.windows.net. Now your clients, be the in Azure or on-premises, can connect to this DNS name/IP address to connect to this Azure SQL instance. What you get:

  • Private access to PaaS services from your on-premises or Azure networks.
  • Access to individual instances of a service, such as an Azure SQL server.
  • (PREVIEW LIMITATIONS) A limited number of platform services in limited US-only regions.

Cloud Camp 2018 – It’s A Wrap!

Yesterday, Cloud Camp 2018, run by MicroWarehouse and sponsored by Microsoft Surface and Veeam, ran in the Dublin Convention Centre here in Ireland. 4 tracks, 20 (mostly MVP) sessions, 2 keynotes, and hundreds of satisfied attendees. It was great fun – but we’re all a little tired today Smile

Photo by Gregor Reimling

The message of the day was “change” and that was what I talked about in the opening keynote. In nature, change is inevitable. In IT, you cannot accept change, you’re pushed aside. Business pressure, security & compliance needs, and the speed of cloud make change happen faster than ever. And that’s why we had 20 expert-lead breakout sessions covering Azure IaaS, Azure PaaS, productivity, security, management & governance, Windows Server 2019 and hybrid cloud solutions. The conference ended with renowned Microsoft-watchers Mary Jo Foley and Paul Thurrott discussing what the corporation has been up to and their experiences in covering the Redmond giant.

We had a lot of fun yesterday. Everything ran quite smoothly – credit to John & Glenn in MWH and Hanover Communications.

After the conference, Paul & Mary Jo hosted their Windows Weekly podcast from Dogpatch Labs in the IFSC.

And then we had a small after party in Urban Brewing next door, where one or two beverages might have been consumed until the wee hours of the morning Smile

Picture by Gerald Versluis

Thank you to:

  • MicroWarehouse for running this event – Rory for OK-ing it and the team for promoting it.
  • John and Glenn who ran the logistics and made it so smooth
  • Hanover Communications for the PR work
  • All the breakout speakers who travelled from around Ireland/Europe to share their knowledge and experience
  • Kartik who travelled from India to share what Azure Backup are up to
  • Paul & Mary Jo for travelling from the USA to spend some time with us
  • Alex at TWiT for make sure things worked well with the podcast
  • Everyone who attended and made this event possible!

A Twitter competition with the #CloudCamp18 tag was run – a winner will be selected (after the dust settles) for a shiny new Surface Go. At one point the #CloudCamp18 tag was trending #3 for tweets in Dublin. Now I wonder what will happen with #CloudCamp19?

AidanFinn.Com Migrated To Azure App Services

I’ve just migrated AidanFinn.com from a Windows Server 2012 R2 Azure virtual machine to an App Service (web app) running on the same App Service Plan as CloudMechanix.com.

Drawing1

This site, AidanFinn.com has been running on an Azure VM for the last few years. That has given me a lot of experience with running a production workload in Azure. Azure worked well. What really irked me was MySQL, running in the VM by the way. MySQL blew up once, and wouldn’t restore, so I had to restore the entire VM. And MySQL continues to be a pain, causing the site to crash, requiring full VM reboots.

I was facing an eventual upgrade of the VM (a migration in Azure) so I made the decision to reduce my maintenance workload. I decided to switch to PaaS, and let Microsoft do the work. I previously blogged how I deployed the Cloud Mexchanix (my Azure training business) onto an Azure app service plan. I also created a stub WordPress site for AidanFinn.com, running on the same plan. The two WordPress site runs on different app services (application pools) on the same WS2016 machine, managed by Microsoft. I have auto-scaling enabled so a single (only in this case) load balanced VM instance can automatically be spun up if the CPU/memory load requires it. Both sites are using Azure Database for MySQL instances, where Microsoft looks after MySQL for me. In other words, the VM, the guest OS, and the database system are managed by Microsoft. I manage the web content. Perfect!

The migration of AidanFinn.com has always been a challenge, from it’s origins as a “Joe Elway” blog on Live Spaces all the way through to it’s previous existence as an Azure VM. I remember the bad old days of exporting and editing multiple XML files to get a migration to work once. And this time was no different. The built-in WordPress Import refused to work. I tried another third-party plug-in and that wouldn’t work. Then I tried the All-In-One WP Migration plugin. It took hours to do a 1 GB export of the content and database from the VM. When I tried to do an import, I exceeded the 512 MB free limit, so I had to pay for the professional edition ($69 or so). The import also took ages, but the site was lifted and shifted exactly as it was.

The it was time to add the custom domain names to the app service in the Azure Portal. A quick query with my DNS registrar (Blacknight) told me how to create @ records in their control panel, and I was done! I will look at hosting the domain in Azure, like I did with Cloud Mechanix, but all the Office 365 records will take time to create first.

image

What about the old machine? It can take up to 24 hours for DNS changes to be replicated around the world, so it will remain running until tomorrow afternoon. I have configured Auto Shutdown in the settings of the VM, with a notification to be sent to me by email first.

And that will be that! Both of my websites will be running on Azure App Services.

The App Service size is S1, costing ~€61.57/month. Each database will cost under €30 per month. Some blob storage (€0.02 per GB) is being used to backup the sites  – restores have been tested! While the total is well above a $10 web hosting plan, I cannot use such plans, because I was kicked off of that platform because AidanFinn.com generated too much load. So it’s either VM or PaaS, and PaaS suits me more because there is less for me to maintain now that I am there.

Speaking at NIC Future Edition 2018

I will be speaking at the NICCONF in Olso, Norway, running 21 Jan to 2 Feb. It’s a big and very well run event, which I was happy to present at last year.

image

I have two sessions:

Forget Virtual Machines – Use Azure Service Fabric For New LOB Apps

This is on Thursday 1st at 10:00 am and puts me right outside my usual comfort zone of IaaS. The subject is PaaS, but hold on IT pros, it’s all based on IaaS which has to be deployed, configured, secured, and monitored. I’ve found Service Fabric to be very interesting because it brings together so many IaaS pieces to create a cool platform for application deployment.

This session, aimed at IT pros (not developers) is an introduction to Service Fabric. I’ll explain what each of the features does, how they can be practically used, and why IT pros should strongly consider using the developer side of Azure for future deployments.

EDIT (Jan 29, 2018): I have built a cool demo environment with Visual Studio (!) and Azure Service Fabric, showing off a “Ticketmaster” that can scale when the likes of Ed Sheeran starts selling tickets, instead of hanging for two hours.

Monitoring Azure IaaS

On Thursday at 13:20, I return to my comfort zone and discuss monitoring your Azure deployment.

In this session I will explain how you can use the various management features of Azure to monitor and alert on the performance & health of your infrastructure deployment in Microsoft Azure.

EDIT (Jan 29, 2018): I have lots of things to show in a demo environment.

Hopefully I’ll see some of you in Oslo in the new year!

 

Would You Like To Learn More About Azure?

If you found this information useful, then imagine what 2 days of training might mean to you. I’m delivering a 2-day course in Amsterdam on April 19-20, teaching newbies and experienced Azure admins about Azure Infrastructure. There’ll be lots of in-depth information, covering the foundations, best practices, troubleshooting, and advanced configurations. You can learn more here.

Video – What is Microsoft Azure?

I’ve posted a short video to help people understand what Microsoft Azure is, how it can impact a business, where it is, how Microsoft has made Azure compliance with lots of regulations and standards, and what Azure can do.

 

image

Was This Video Useful?

If you found this information useful, then imagine what 2 days of training might mean to you. I’m delivering a 2-day course in Amsterdam on April 19-20, teaching newbies and experienced Azure admins about Azure Infrastructure. There’ll be lots of in-depth information, covering the foundations, best practices, troubleshooting, and advanced configurations. You can learn more here.

Overview of the Microsoft Azure Serverless Platform

Speakers:

  • Chris Anderson, Senior Program Manager
  • Raman Sharma, Senior Product Marketing Manager

Evolution of Application Platforms

We used to build applications on-premises. Find h/w, find someone to set it up, and then the bits, networking, etc. Counter to productivity.

IaaS meant that a VM could be provisioned faster, but still left with some delays, and still required on-going management. The current wave of SaaS is probably mostly built on IaaS.

Along came PaaS. VMs were abstracted, hiding the on-going management of the VM and guest OS. But we still have the same per-VM paradigm.

Serverless ideally reduces infrastructure to zero. There is really a server, but it’s not yours and there’s no hint of it. The app still has to run on something, but you never acknowledge its existence.

Traits of Serverless

  • Abstraction of servers
  • Event-driven start-up triggers/instant scale out or in.
  • Micro-billing instead of per-hour/month billing.

Benefits

  • Manage apps, not servers. Look after the thing you care about: the app and the code.
  • Reduced DevOps: there are no servers to fix/patch. You don’t have to scale out traditional PaaS.
  • Faster time to market

Your business can sell a service, consume data, seize an opportunity quicker than ever with serverless.

Serverless Application Platform Components

  • Functions: Execute your code based on events you specify.
  • Logic Apps: Design workflows and orchestrate processes. It’s the original serverless feature in Azure. Logic Apps tie together different things inside and outside of Azure in workflows made up of different steps.
  • Event Grid: A relatively new service in Public Preview. Manages all events that can trigger code or logic. A single interface for all such events. Manage where those events come from, and decide what you will do with those events (start Functions, Logic Apps, etc)

26-09-2017 16-18 Office Lens

  • Database: store the data
  • Storage: store data in blob/queue/NoSQL
  • Security services
  • IoT for massive numbers of devices inputing data
  • Analytics to process data realtime
  • Intelligence to use AI models to understand/action on data

Demo: Event Grid & Logic Apps (Chris)

In the Logic Apps Designer. Looks like Flow. When an event occurs to a storage account and is sent in from Event Grid, a post is sent to Teams. The body of the message is dumped into Teams, which is ugly JSON. They want to parse the JSON before posting it. A function is written in JavaScript. The function will do the required parsing of the JSON and create human-friendly output.

He adds an action between the two existing actions in the logic app designer. This will trigger the Function. The Function is passed the body of the JSON. The Function will return a response, and that is used as an input to the Teams action. That input is given a condition. If verbosity = ignore then the Teams action can be bypassed.

Now he deletes a storage account and Event Grid starts the logic app. He traces the logic app in the Portal job history – the trace shows that Team received the message from Logic Apps. After some debugging, the formatted event appeared in Teams. Quite a bit was accomplished in that process without a VM and with very little JavaScript.

Scenarios for Serverless

Short: Anything that is based on events.

  • Real-time stream processing, e.g. IoT. You have no idea when data is coming in, and how much will come in.
  • Time-based processing. E.g. collect logs and process them once per week. Why dedicate VMs/PaaS for that? Pay for the few seconds of compute that is required for the task.
  • Back-ends for mobile/IoT/web. E.g. someone uploads an image to a site, a function stores the image in a storage account, and another function processes the image (thumbnails, metadata, etc) and stores that data somewhere.
  • Real-time bot messaging. Use Logic Apps to workflow the processing of a question from a bot in Cortana Analytics and then send the result back to the bot.

Event Grid

26-09-2017 16-36 Office Lens

The list of services will grow – it’s only been in preview for a month and a half at this point.

It delivers at massive scale:

  • Sub-second end-to-end latency in 99th percentile
  • 10,000,000 events per second per region.
  • 24-hour retry with exponential back off for events not delivered

Logic Apps

  • A visual design experience without writing any code, no matter how complex it is.
  • Multiple connectors to Azure, third-party, or your own services/functions.
  • Uses a declarative definition format to work with CI/CD.

There are over 170 (and growing) connectors to orchestrate, e.g. Slack, SalesForce, Twitter, Box, Facebook, GitHub, DropBox, Pintrest, WordPress, etc.

Functions

An evolution of WebJobs from PaaS.

  • Develop locally using best of class developer tools
  • Boost productivity through triggers and bindings.
  • Choose from a variety of programming languages
  • Integrate with existing DevOps processes.

Lots of triggers: schedule, HTTP (REST or webhook), Blob storage, events, queues, queues and topics, storage tables, SQL tables, NoSQL DB, Push Notifications, Wwilio SMS Text … and one I missed.

What’s New in Functions

  • You can develop locally on Linux, MacOS, and Linux
  • Monitor serverless applications using Application Insights (now GA)
  • Trigger a function on changes in Cosmos DB
  • Securely provide access to information in Microsoft Graph through a function.
  • Trigger a function from a real-time analytics pipeline in Stream Analytics

Demo

Four volunteers go on stage. There are two IoT buttons. When they are pressed, they’ll send a message to Azure IoT, and will trigger Functions. Fastest to answer questions get most points, last gets least points. And then questions on Serverless tech come up. The scores are processed by Serverless compute using Functions. In this case, the functions were actually running on a local host (WS2016).

Customer Stories

  • Fujifilm: Online service handling 1 TB data per day. Monolithic design where code change was hard. Cosmos DB and Functions reduced latency by 95%. Development time reduced by 75%. Higher reliability. Ability to add new features and release faster and more frequently.
  • Quest: Uses serverless for a SaaS application. Able to manage millions of objects. Cut time-to-market by 2/3. No costly on-premises h/w.
  • Plexure: Service to help retailers understand “signals” from their stores to optimize sales. Built serverless with functions, even hubs, IoT, machine learning and Cosmos DB. Reduced efforts and scales on demand. Used to build infrastructure for each retailer. Now they just publish APIs.