The Azure IaaS Book Of News – December 2022

Here’s all the news that I thought was interesting for Ops and Security folks working with Azure IaaS from December 2022.

Azure VMware Solution

  • Azure VMware Solution Advanced Monitoring: This solution add-on deploys a virtual machine running Telegraf in Azure with a managed identity that has contributor and metrics publisher access to the Azure VMware Solution private cloud object. Telegraf then connects to vCenter Server and NSX-T Manager via API and provides responses to API metric requests from the Azure portal.

Azure Kubernetes Service

  • Microsoft and Isovalent partner to bring next generation eBPF dataplane for cloud-native applications in Azure: Microsoft announces the strategic partnership with Isovalent to bring Cilium’s eBPF-powered networking data plane and enhanced features for Kubernetes and cloud-native infrastructure. Azure Kubernetes Services (AKS) will now be deployed with Cilium open-source data plane and natively integrated with Azure Container Networking Interface (CNI). Microsoft and Isovalent will enable Isovalent Cilium Enterprise as a Kubernetes container App offering onto Azure Container Marketplace. This will provide a one-click deployment solution to Azure Kubernetes clusters with Isovalent Cilium Enterprise advanced features.
  • Generally Available: Kubernetes 1.25 support in AKS: AKS support for Kubernetes release 1.25 is now generally available. Kubernetes 1.25 delivers 40 enhancements. This release includes new changes such as the removal of PodSecurityPolicy.

Azure Backup

Azure Virtual Desktop

Virtual Machines

  • Public preview: New Memory Optimized VM sizes – E96bsv5 and E112ibsv5: The new E96bsv5 and E112ibsv5 VM sizes part of the Azure Ebsv5 VM series offer the highest remote storage performances of any Azure VMs to date.  The new VMs can now achieve even higher VM-to-disk throughput and IOPS performance with up to 8,000 MBps and 260,000 IOPS.
  • Generally Available: Azure Dedicated Host – Restart: Azure Dedicated Host gives you more control over the hosts you deployed by giving you the option to restart any host. When undergoing a restart, the host and its associated VMs will restart while staying on the same underlying physical hardware.

Governance

  • Public preview: Use tag inheritance for cost management: You no longer need to ensure that every resource is tagged or rely on resource providers to support and emit tags in their billing pipeline for cost management. Aidan’s Note – Restricted to EA/MCA … which unreasonably sucks. The latest example of “cost management” excluding other customers.

App Services

Storage

Azure Site Recovery

  • Public Preview: Azure Site Recovery Higher Churn Support: Azure Site Recovery (ASR) has increased its data churn limit by approximately 2.5x to 50 MB/s per disk. With this, you can configure disaster recovery (DR) for Azure VMs having data churn up to 100 MB/s. This helps you to enable DR for more IO intensive workloads.

Networking

Miscellaneous

Your Azure Migration Project Is Doomed To Fail

That heading is a positive way to get your holiday spirit going, eh? I’m afraid I’m going to be your Festive Tech Calendar 2022 Grinch and rain doom and gloom on you for the next while. But hang on – The Grinch story ends with a happy-ever-after.

In this article, I want to explain why a typical “Azure migration” project is doomed to fail to deliver what the business ultimately expects: an agile environment that delivers on the promises of The Cloud. The customers that are affected are typically larger: those with IT operations and one or more development teams that create software for the organisation. If you don’t fall into that market, then keep reading because this might still be interesting to you.

My experience is Microsoft-focused, but the tale that I will tell is just as applicable to AWS, GCP, and other alternatives to Microsoft Azure.

Migration Versus Adoption

I have been working with mid-large organisations entering The Cloud for the last 4 years. The vast majority of these projects start with the same objectives:

  1. Migrate existing workloads into Azure, usually targeting migration to virtual machines because of timelines and complexities with legacy or vendor-supplied software.
  2. Build new workloads and, maybe, rearchitect some migrated workloads.

Those two objectives are two different projects:

  1. Migration: Get started in Azure and migrate the old stuff
  2. Adoption: Start to use Azure to build new things

Getting Started

The Azure journey typically starts in the IT Operations department. The manager that runs infrastructure architecture/operations will kick off the project either in-house or with a consulting firm. Depending on who is running things, there will be some kind of process to figure out how to prepare the landing zone(s) (the infrastructure) in Azure, some form of workloads assessment will be performed, and existing workloads will be migrated to The Cloud. That project, absent of (typically organisation) issues, should end well:

  • The Azure platform is secure and stable
  • The old workloads are running well in their landing zone(s)

The Migration project is over, there is a party, people are happy, and things are … great?

Adoption Failure

Adoption starts after Migration ends. Operations has built their field of dreams and now the developers will come – cars laden with Visual Studio, containers, code, and company aspirations will be lining up through the corn fields of The Cloud, eager to get going … or not. In reality, if Migration has happened like I wrote above, things have already gone wrong and it is already probably too late to change the path forward. That’s been my experience. It might take 1 or 2 years, but eventually, there will be a meeting where someone will ask: why do the developers not want to use Azure?

The first case of this that I saw happened with a large multinational. Two years after we succeeded with a successful Azure deployment – the customer was very happy at the time – we were approached by another department, the developers, in the company. They loved what we built in Azure but they hated how it was run by IT. IT had taken all their ITIL processes, ticketing systems, and centralised control that were used for on-premises and applied them to what we delivered in Azure. The developers were keen to use agile techniques, tools, and resource types in Azure, but found that the legacy operations prevented this from working. They knew who built the Azure environment, they came to us, and asked us to build a shadow IT environment in Azure just for them, outside of the control of IT Operations!

That sounds crazy, right? But this is not a unique story. We’ve seen it again and again. I hear stories of “can we create custom roles to lock down what developers can do?” and then that adoption is failing. Or “developers find it easier to work elsewhere”. What has gone wrong?

Take a Drink!

Adoption is when the developers adopt what is delivered in Azure and start to build and innovate. They expect to find a self-service environment that enables cool things like GitOps, Kubernetes, and all those nerdy cool things that will make their job easier. But their adoption will not happen because they do not get that. They get a locked-down environment where permissions are restricted, access to platform resources might be banned because Operations only know & support Windows, and the ability to create firewall rules for Internet access or to get a VM is hidden behind a ticketing system with a 5-day turnaround.

Some of us in the community joke about taking a drink every time some Microsoft presenter says a certain phrase at a conference. One of those phrases in the early days of Microsoft Azure was “cloud is not where you work, it is how you work”. That phrase started reappearing in Microsoft events earlier this year. And it’s a phrase that I have been using regularly for nearly 18 months.

The problem that these organisations have is that they didn’t understand that how IT is delivered needs to change in order to provide the organisation with the agile platform that the Cloud can be. The Cloud is not magically agile; it requires that you:

  • Use the right tools & technology, which involves skills acquisition/development
  • People are reorganised
  • New processes are implemented

Let’s turn that into the Cloud Equation:

Cloud = Process(People + Tools)

Aidan Finn, failed mathematician and self-admitted cloud Grinch

With the current skills shortage, one might think that the tools/tech/skills part is the hard part of the Cloud Equation. Yes; developing or acquiring skills is not easy. But, that is actually the easiest part of the Cloud Equation.

Our business has had a split that has been there since the days of the first computer. It is worse than Linux versus Windows. It is worse than Microsoft versus Apple. It is worse than ex-Twitter employees versus Elon. The split that I talk of is developers (Devs) versus operations (Ops). I’ve been there. I have been the Ba*tard Operator From Hell – I took great joy in making the life of a developer miserable so that “my” systems were stable and secure (what I mistakenly viewed as my priority). I know that I was not unique – I see it still happens in lots of organisations.

People must be reorganised to work together, plan together, build and secure together, to improve workloads together … together. Enabling that “together” requires that old processes are cast aside with the physical data centre, and new processes that are suitable for The Cloud are used.

Why can’t all this be done after Migration has been completed? I can tell you that for these organisations, the failure happened a long time before Migration started.

The Root Cause

According to my article, who initiated the cloud journey? It was a manager in IT Operations. Who owns the old processes? The manager in IT Operations. Who is not responsible for the developers? The manager in IT Operations. Who is not empowered, and maybe not motivated, to make the required Tools(People + Process) changes that can enable an agile Cloud.

In my experience, a cloud journey project will fail at the Adoption phase if the entire journey does not start at the board level. Someone in the C-suite needs to kick that project off – and remain involved.

A Solution

Mcrosoft Cloud Adoption Framework 2022
Microsoft Cloud Adoption Framework 2022

The Microsoft Cloud Adoption Framework (AKA CAF and AWS and GCP have their own versions) recognises that  Cloud Journey must start at the top. The first thing that should be done is to sit down with organisation leadership to discuss and document business motivations. The results of the conversation(s) should result in:

  1. A documented cloud strategy plan
  2. A clear communication to the business: this is how things will be with the implied “or else”

Only the CEO, CTO (or equivalent) can give the order that Operations and Development must change how things are done DevOps. Ideally, Security is put into that mix considering all the threats out there and that Security should be a business priority: DevSecOps. Only when all this is documented and clearly communicated, should the rest of the process start.

The Plan Phase of the CAF has some important deliverables:

  • Initial organisation alignment: Organise staff with their new roles in DevSecOps – The People part of the Cloud Equation.
  • Skills Readiness: Create the skills for the Tools part of the Cloud Equation.

Engineering starts in the Ready phase; this is when things get turned on and the techies (should) start to enjoy the project. The very first phase in the recently updated CAF is the Operating Model; this is a slight change that recognises the need for process change. This is when the final part of the Cloud Equation can be implemented: the new processes multiply the effect of tools/skills and organisation. With that in place, we should be ready, as a unified organisation, to build things for the organisation – not for IT Operations or the Developers – with agility and security.

I have skipped over all the governance, management, compliance, and finance stuff that should also happen to highlight the necessary changes that are usually missing.

Whether the teams are real or virtual, in the org chart or not, each workload will have a team made up of operations, security, and development skills. That team will share a single backlog that starts out with a mission statement or epic that will have a user story that says something like “the company requires something that does XYZ”. The team will work together to build out that backlog with features and tasks (swap in whatever agile terminology you want) as the project progresses to plan out tasks across all the required skill sets in the team and with external providers. This working together means that there are no surprises, Devs aren’t “dumping” code on IT, IT isn’t locking down Devs, and Security is built-in and not a hope-for-the-best afterthought.

Final Thoughts

The problem I have described is not unique. I have seen it repeatedly and it has happened and is happening all around the world. The fix is there. I am not saying that it is easy. Getting access to the C-Level is hard for various reasons. Changing momentum that has been there for decades is a monumental mission. Even when everyone is aligned, the change will be hard and take time to get right but it will be rewarding.  Bon, voyage!

Ignite 2022 IaaS Blog Post Of News

This post is my alternative to the Microsoft Ignite “Book of News”.

You’ve probably heard of or even read the Ignite Book of News. This is a PDF that is sent out to those under NDA (media, MVPs, and so on) before Microsoft Ignite starts. After the kickoff, the document is shared publicly. The Book of News is heavily shaped by Marketing, focusing on highlights and the “message” of the conference. The Book of News is not complete, despite all claims by those who are poorly informed – over the years, I’ve found countless announcements from sessions and product group blog posts that were not in the Book of News.

I’m taking part in an “Ignite After Party” to discuss the Book of News. The organiser has encouraged going “off book” so I’ve summarised all the IaaS stuff that I could find (and a little PaaS) – most of this stuff was not in the Book of News. Here you will find all the announcements in that space from Ignite and the time since then (I stopped at November 30th when I wrote this post).

Ignite News

App Services

Go available on App Service

https://azure.github.io/AppService/2022/10/12/Go-on-AppService.html

We are happy to announce that App Service now supports apps targeting Go 1.18 and 1.19 across all public regions on Linux App Service Plans through the App Service Early Access feature. By introducing native support for Go on App Services, we are making one of the top 10 best loved web app development languages available for our developers.

In development: Larger SKUs for App Service Environment v3

https://azure.microsoft.com/en-gb/updates/in-development-larger-skus-for-app-service-environment-v3/

New Isolated v2 SKUs of 64GB/ 128GB/ 256GB provide compelling value to organizations that need a dedicated tenant to run their most sensitive and demanding applications. This is expected to be available in production in Q4 CY2022.

Public preview: Planned maintenance feature for App Service Environment v3

http://azure.microsoft.com/en-us/updates/public-preview-planned-maintenance-feature-for-app-service-environment-v3/

With planned maintenance notifications for App Service Environment v3, you can get a notification 15 days ahead of planned automatic maintenance and start the maintenance when it is convenient for you

Hybrid

Announcing Jumpstart ArcBox for DataOps

https://techcommunity.microsoft.com/t5/azure-arc-blog/announcing-jumpstart-arcbox-for-dataops/ba-p/3647642

ArcBox for DataOps, is our road-tested automation providing our customers a way to get hands-on with the Azure Arc-enabled SQL Managed Instance set of capabilities and features.

Announcing Jumpstart HCIBox

https://techcommunity.microsoft.com/t5/azure-arc-blog/announcing-jumpstart-hcibox/ba-p/3647646

HCIBox is a turnkey solution that provides a complete sandbox for exploring Azure Stack HCI capabilities and hybrid cloud integration in a virtualized environment. HCIBox is designed to be completely self-contained within a single Azure subscription and resource group, which will make it easy for a user to get hands-on with Azure Stack HCI and Azure Arc technology without the need for physical hardware.

CAF

Announcing Landing Zone Accelerator for Azure Arc-enabled SQL Managed Instance

https://techcommunity.microsoft.com/t5/azure-arc-blog/announcing-landing-zone-accelerator-for-azure-arc-enabled-sql/ba-p/3647623

a proven set of guidance designed by subject matter experts across Microsoft to help customers create and implement the business and technology strategies necessary to succeed in the cloud as well as a way to automate a fully deployed Azure Arc-enabled SQL Managed Instance environment, making implementation faster.

AVD

Announcing general availability of support for Azure availability zones in the host pool deployment

https://techcommunity.microsoft.com/t5/azure-virtual-desktop-blog/announcing-general-availability-of-support-for-azure/ba-p/3636262

I am pleased to announce that you can now automatically distribute your session hosts across any number of availability zones

New ways to optimize flexibility, improve security, and reduce costs with Azure Virtual Desktop

https://techcommunity.microsoft.com/t5/azure-virtual-desktop-blog/new-ways-to-optimize-flexibility-improve-security-and-reduce/ba-p/3650895

With the public preview of new integrations with Azure Active Directory, you can use single sign-on and passwordless authentication, leveraging FIDO2 standards and Windows Hello for Business to securely streamline the authentication experience for today’s remote and hybrid workforce.

Now in public preview, customers can use cloud storage to host FSLogix and modern Azure Active Directory authentication for their session hosts (more on that later).

Public preview for confidential virtual machine options for Azure Virtual Desktop is also available now—specifically for Windows 11 virtual machines—with Windows 10 support planned in the future.

customers who require their information to remain on trusted private networks will have the option to use Private Link to enable access to their session hosts and workspaces over a private endpoint in their virtual network.

Cost Management

Optimize and maximize cloud investment with Azure savings plan for compute

https://techcommunity.microsoft.com/t5/azure-compute-blog/optimize-and-maximize-cloud-investment-with-azure-savings-plan/ba-p/3636447

Today, we are announcing Azure savings plan for compute. With this new pricing offer, customers will have an easy and flexible way to save up to 65%* on compute costs, compared to pay-as-you-go pricing, in addition to existing offers in market including Azure Hybrid Benefit and Reservations.

Storage

General availability: Azure Premium SSD v2 Disk Storage

http://azure.microsoft.com/en-us/updates/general-availability-azure-premium-ssd-v2-disk-storage/

In summary, Premium SSD v2 offers the following key benefits:

  • Ability to increase disk storage capacity in 1 GiB increments.
  • The capability to separately provision IOPS, throughput, and disk storage capacity.
  • Consistent sub-millisecond latency.
  • Easier maintenance with scaling performance up and down without downtime.
  • Up to 64TiBs, 80,000 IOPS and 1200 MB/s on a single disk.

Public preview: Azure Elastic SAN

http://azure.microsoft.com/en-us/updates/public-preview-azure-elastic-san/

With Elastic SAN, you can deploy, manage, and host workloads on Azure with an end-to-end experience similar to on-premises SAN. The solution also enables bulk provisioning of block storage that can achieve massive scale, up to millions of IOPS, double-digit GB/s of throughput, and low single-digit millisecond latencies with built-in resiliency to minimize downtime.

Management

Generally available: Azure Automanage for Azure Virtual Machines and Arc-enabled servers

https://azure.microsoft.com/en-gb/updates/generally-available-azure-automanage-for-azure-virtual-machines-and-arcenabled-servers/

Azure Automanage is a service that automates configuration of virtual machines (VMs) to best-practice Azure services, as well as continuous security and management operations across the entire lifecycle of VMs in Azure or hybrid environments enabled through Azure Arc. This allows you to save time, reduce risk, and improve workload uptime by automating day-to-day configuration and management tasks– all with point-and-click simplicity, at scale.

Generally available: Azure Monitor agent support for Windows clients

http://azure.microsoft.com/en-us/updates/generally-available-azure-monitor-agent-support-for-windows-clients/

The Azure Monitor agent and data collection rules now support Windows 10 and 11 client devices via the new Windows MSI installer. Extend the use of the same agent for telemetry and security management (using Sentinel) across your service and device landscape.

Generally available: Azure Monitor agent migration tools

http://azure.microsoft.com/en-us/updates/generally-available-azure-monitor-agent-migration-tools/

Per earlier communication, you must migrate from log analytics agent (MMA or OMS agents) to this agent before August 2024. You can use agent migration tools now generally available to make this process easier for you.

Public preview: Azure Monitor Logs – create granular level RBAC for custom tables

https://azure.microsoft.com/en-gb/updates/public-preview-azure-monitor-logs-create-granular-level-rbac-for-custom-tables/

The Log Analytics product team added two additional capabilities to enable workspace admins to manage more granular data access, supporting read permission at the table level both for Azure tables and customer tables.  

Cost-effective solution for high-volume verbose logs

https://techcommunity.microsoft.com/t5/azure-observability-blog/general-availability-get-more-value-from-your-logs-with-azure/ba-p/3643129

Basic Logs is a new flavor of logs that enables a lower-cost collection of high-volume verbose logs that you use for debugging and troubleshooting, but not for analytics and alerts. This data, which might have been historically stored outside of Azure Monitor Logs, can now be available inside your Log Analytics workspace, enabling one solution for all your log data.

Low-cost long-term storage of your log data

https://techcommunity.microsoft.com/t5/azure-observability-blog/general-availability-get-more-value-from-your-logs-with-azure/ba-p/3643129

Log Archive is an in-place solution to store your data for long-term retention of up to seven years at a cost-effective price point. This lets you store all your data in Azure Monitor Logs, without having to manage an external data store for archival purposes, and query or import data in and out of Azure Monitor Logs. You can access archived data by running a search job or restoring it for a limited time for investigation, as detailed below. 

Search through large volumes of log data

https://techcommunity.microsoft.com/t5/azure-observability-blog/general-availability-get-more-value-from-your-logs-with-azure/ba-p/3643129

A search job can run from a few minutes to hours, scanning log data and fetching the relevant records into a new persistent search job results table. The search job results table supports the full set of analytics capabilities to enable further analysis and investigation of these records.

Investigate archived logs

https://techcommunity.microsoft.com/t5/azure-observability-blog/general-availability-get-more-value-from-your-logs-with-azure/ba-p/3643129

Restore is another tool for investigating your archived data. Unlike the search job, which accesses data based on specific criteria, restore makes a given time range of the data in a table available for high-performance queries. Restore is a powerful operation, with a relatively high cost, so it should be used in extreme cases when you need direct access to your archived data with the full interactive range of analytics capabilities.

Generally available: Windows Admin Center for Azure Virtual Machines

https://azure.microsoft.com/en-gb/updates/windows-admin-center-for-azure-virtual-machines/

Windows Admin Center lets you manage the Windows Server Operating System of your Azure Virtual Machines, natively in the Azure Portal. You can perform maintenance and troubleshooting tasks such as managing your files, viewing your events, monitoring your performance, getting an in-browser RDP and PowerShell session, and much more, all within Azure.

Set up alerts faster with our new and simplified alerting experience (in preview)

https://techcommunity.microsoft.com/t5/azure-observability-blog/what-s-new-in-azure-monitor-ignite-2022/ba-p/3652570

Recommended alert rules provides customers with an easy way to enable a set of best practice alert rules on their Azure resources. This feature, which previously supported only virtual machines, is now being extended to also support AKS and Log Analytics Workspace resources.

Azure VMware Solution

Public preview: Customer-managed keys for Azure VMware Solution

https://azure.microsoft.com/en-gb/updates/public-preview-customermanaged-keys-for-azure-vmware-solution/

Customer-managed keys (CMK) for Azure VMware Solution (AVS) provides you with control over your encrypted vSAN data on Azure VMware Solution. With this feature, you can use Azure Key Vault to generate customer-managed keys as well as centralize and streamline the key management process.

Public preview: Stretched clusters for Azure VMware Solution

http://azure.microsoft.com/en-us/updates/public-preview-stretched-clusters-for-azure-vmware-solution/

provides 99.99% uptime for mission critical applications. Stretched cluster benefits:

  • Improve application availability.
  • Provide a zero-recovery point objective (RPO) capability for enterprise applications without needing to redesign or deploy expensive disaster recovery (DR) solutions.
  • A private cloud with stretched clusters is designed to provide 99.99% availability due to its resilience to availability zone failures.
  • Enables you to focus on core application requirements and features, instead of infrastructure availability.

AKS

Generally available: Azure Hybrid Benefit for AKS and Azure Stack HCI

http://azure.microsoft.com/en-us/updates/generally-available-azure-hybrid-benefit-for-aks-and-azure-stack-hci/

At Ignite, we are expanding Azure Hybrid Benefit to further reduce costs for on-premises and edge locations. Customers with Windows Server Software Assurance (SA) can use Azure Hybrid Benefit for Azure Kubernetes Service (AKS) and Azure Stack HCI to:

  • Run AKS on Windows Server and Azure Stack HCI at no additional cost in datacenter and edge locations. With this, you can deploy and manage containerized Linux and Windows applications from cloud to edge with a consistent, managed Kubernetes service. This applies to Windows Server Datacenter and Standard Software Assurance and Cloud Solution Provider (CSP) customers.
  • Use first-party Arc-enabled infrastructure, Azure Stack HCI, at no additional cost. Windows Server Datacenter Software Assurance customers can modernize their existing datacenter and edge infrastructure to run their VM and container-based applications on modern infrastructure with industry-leading price-performance and built-in connectivity to Azure.

Public preview: Azure Kubernetes Service hybrid deployment options

https://azure.microsoft.com/en-gb/updates/public-preview-azure-kubernetes-service-hybrid-deployment-options/

Azure Kubernetes Service (AKS) on Azure Stack HCI, Windows Server 2019, and 2022 Datacenter can be provisioned from the Azure Portal/CLI. Additionally, AKS is now in public preview on Windows devices and Windows IoT for lightweight Kubernetes orchestration.

Generally available: 5,000 node scale in AKS

http://azure.microsoft.com/en-us/updates/generally-available-5000-node-scale-in-aks/

Azure Kubernetes Service is increasing the maximum node limit per cluster from 1,000 nodes to 5,000 nodes for customers using the uptime-SLA feature.

Generally available: Windows server 2022 host support in AKS

https://azure.microsoft.com/en-gb/updates/generally-available-windows-server-2022-host-support-in-aks/

With this generally available feature, Windows Server 2022 is now supported on AKS. Among other improvements related to security, Windows Server 2022 also provides several platform improvements for Windows Containers and Kubernetes. Windows Server 2022 is available for Kubernetes v1.23 and higher.

Public preview: Kubernetes apps on Azure Marketplace

http://azure.microsoft.com/en-us/updates/public-preview-kubernetes-apps-on-azure-marketplace/

You can now browse the catalog of solutions specialized for Kubernetes platforms under Kubernetes apps offer in marketplace and select a solution for click through deployment to Azure Kubernetes Service (AKS) with automated Azure billing.

Public preview: Azure CNI Overlay mode in Azure Kubernetes Service

https://azure.microsoft.com/en-gb/updates/public-preview-azure-cni-overlay-mode-in-azure-kubernetes-service/

Azure CNI Overlay mode is a new CNI network plugin that allocates pod IPs from an overlay network space, rather than from the virtual network IP space.

General availability: AMD-based confidential VMs for Azure Kubernetes Service

http://azure.microsoft.com/en-us/updates/general-availability-amdbased-confidential-vms-for-azure-kubernetes-service/

With the general availability of confidential virtual machines featuring AMD 3rd Gen EPYC™ processors, with Secure Encrypted Virtualization-Secure Nested Paging (SEV-SNP) security features, organizations get VMs with isolated, encrypted memory, and genuine confidentiality attestation rooted to the hardware.

Public preview: Rules for Azure Kubernetes Service and Log Analytic workspace resources

https://azure.microsoft.com/en-gb/updates/public-preview-rules-for-azure-kubernetes-service-and-log-analytic-workspace-resources/

Enable a set of best practice alert rules on an unmonitored AKS and Log Analytics workspace resource with just a few clicks.

Public preview: Azure Monitor managed service for Prometheus

http://azure.microsoft.com/en-us/updates/public-preview-azure-monitor-managed-service-for-prometheus/

The new fully managed Prometheus compatible service from Azure Monitor delivers the best of what you like about the open-source ecosystem while automating complex tasks such as scaling, high-availability, and long-term data retention. It is available to use as a standalone service from Azure Monitor or as an integrated component of Container Insights and Azure Managed Grafana.

Generally available: ARM64 support in AKS

http://azure.microsoft.com/en-us/updates/generally-available-arm64-support-in-aks/

Announcing the general availability of ARM64 node pool support in AKS. ARM64 provides a better price and compute comparison due to its lower power utilization.

Networking

Public preview: IP Protection SKU for Azure DDoS Protection

https://azure.microsoft.com/en-gb/updates/public-preview-ip-protection-sku-for-azure-ddos-protection/

Instead of enabling DDoS protection on a per virtual network basis, including all public IP resources associated with resources in those virtual networks, you now have the flexibility to enable DDoS protection on an individual public IP.

General availability: Azure DNS Private Resolver – hybrid name resolution and conditional forwarding

http://azure.microsoft.com/en-us/updates/general-availability-azure-dns-private-resolver-hybrid-name-resolution-and-conditional-forwarding/

Azure DNS Private Resolver is a cloud-native, highly available, and DevOps-friendly service. It provides a simple, zero- maintenance, reliable, and secure DNS service to resolve and conditionally forward DNS queries from a virtual network, on-premises, and to other target DNS servers without the need to create and manage a custom DNS solution

WordPress on Azure App Service supports Azure Front Door Integration

https://azure.github.io/AppService/2022/10/12/Announcing-Preview-of-Azure-Front-Door-integration-with-Azure-App-Service.html

We are happy to announce the preview of WordPress on Azure App Service powered by Azure Front Door which enables faster page loads, enhanced security, and increased reliability for your global apps with no configuration or additional code required.

General availability: Custom network interface name configurations of private endpoints

https://azure.microsoft.com/en-gb/updates/general-availability-custom-nic-name-configurations-of-private-endpoints/

This feature allows you to define your own string name at the time of creation of the private endpoint NIC deployed.

General availability: Static IP configurations of private endpoints

http://azure.microsoft.com/en-us/updates/general-availability-static-ip-configurations-of-private-endpoints/

This feature allows you to add customizations to your deployments. Leverage already reserved IP addresses and allocate them to your private endpoint without relying on the randomness of Azure’s dynamic IP allocation.

Public preview: ExpressRoute Traffic Collector

http://azure.microsoft.com/en-us/updates/public-preview-expressroute-traffic-collector/

ExpressRoute Traffic Collector enables sampling of network flows sent over your ExpressRoute Direct circuits. Flow logs get sent to a Log Analytics workspace where you can create your own log queries for further analysis, export the data to any visualization tool or SIEM (Security Information and Event Management) of your choice

In development: Introducing ExpressRoute Metro

https://azure.microsoft.com/en-gb/updates/in-development-introducing-expressroute-metro/

ExpressRoute Metro offers you the ability to create private connections via an ExpressRoute Circuit with dual connections from a Service provider (AT&T, Equinix, Verizon etc.,) or connecting directly with ExpressRoute Direct over a dual 10 Gbps or 100 Gbps physical port in two different Microsoft Edge location in a metropolitan area offering higher redundancy and resiliency. 

Virtual Machines

General availability: New Azure proximity placement groups feature

https://azure.microsoft.com/en-gb/updates/ppgintent/

With the addition of the new optional parameter, intent, you can now specify the VM sizes intended to be part of a proximity placement group when it is created. An optional zone parameter can be used to specify where you want to create the proximity placement group. This capability allows the proximity placement group allocation scope (datacenter) to be optimally defined for the intended VM sizes, reducing deployment failures of compute resources due to capacity unavailability.

General availability: Confidential VM option for SQL Server on Azure Virtual Machines

https://azure.microsoft.com/en-gb/updates/general-availability-confidential-vm-option-for-sql-server-on-azure-virtual-machines/

With the confidential VM option for SQL Server on Azure Virtual Machines, you can now run your SQL Server workloads on the latest AMD-backed confidential virtual machines.

General availability: AMD confidential VM guest attestation

http://azure.microsoft.com/en-us/updates/general-availability-amd-confidential-vm-guest-attestation/

It lets you do the following:

  • Use the guest attestation feature to verify that a confidential VM is running on a hardware-based trusted execution environment (TEE) with security features (isolation, integrity, secure boot) enabled.
  • Allow application deployment decisions (whether to launch a sensitive workload) based on the hardware state returned by the library call.
  • Use remote attestation artifacts (token and claims) received from another system (on a confidential VM) to enable relying parties to gain trust to make transactions with the other system.
  • Receive recommendations and alerts of unhealthy confidential VMs in Microsoft Defender for Cloud.

Announcing the new Ebsv5 VM sizes offering 2X remote storage performance with NVMe-Public Preview

https://techcommunity.microsoft.com/t5/azure-compute-blog/announcing-the-new-ebsv5-vm-sizes-offering-2x-remote-storage/ba-p/3652000

Today, we are announcing the Public Preview of two additional Virtual Machine (VM) sizes, E96bsv5 and E112ibsv5, to the Ebsv5 VM family. The two new sizes are developed with the NVMe protocol and provide exceptional remote storage performance offering up to 260,000 IOPS and 8,000 MBps throughput.

General availability: Azure Monitor predictive autoscale for Azure Virtual Machine Scale Sets

https://azure.microsoft.com/en-gb/updates/general-availability-azure-monitor-predictive-autoscale-for-azure-virtual-machine-scale-sets/

Predictive autoscale uses machine learning to help manage and scale Azure Virtual Machine Scale Sets with cyclical workload patterns. It forecasts overall CPU load to your virtual machine scale set based on your historical CPU usage patterns. By observing and learning from historical usage, it predicts the overall CPU load ensuring scale-out occurs in time to meet demand.

Miscellaneous

Public preview: Microsoft Azure Deployment Environments

https://azure.microsoft.com/en-gb/updates/public-preview-microsoft-azure-deployment-environments/

Azure Deployment Environments has entered public preview. Azure Deployment Environments help dev teams create and manage all types of environments throughout the application lifecycle with features such as:

  • On-demand environments enable developer to spin up environments with each feature branch to enable higher quality code reviews and ensure devs can view and test their changes in a prod-like environment. 
  • Sandbox environments can be used as greenfield environments for experimentation and research.
  • CI/CD pipeline environments integrate with your CI/CD deployment pipeline to automatically create dev, test (regression, load, integration), staging and production environments at specified points in the development lifecycle.
  • Environment types enable dev infra and IT teams to create preconfigured mappings that automatically apply the right subscriptions, permissions and identities to environments deployed by developers based on their current stage of development.
  • Template catalogues housed in a code repo that can be accessed and edited by developers and IT admins to propagate best practices while maintaining security and governance.

Generally available: Azure Site Recovery update rollup 64 – October 2022

https://azure.microsoft.com/en-gb/updates/generally-available-azure-site-recovery-update-rollup-64-october-2022/

Modernized VMware to Azure DR is now generally available.  Added support for: 

  • Protecting physical machines using the modernized experience.
  • Enabling modernized experience with managed identity and private endpoint turned on.

Azure PowerShell Ignite 2022 announcements

https://techcommunity.microsoft.com/t5/azure-tools-blog/azure-powershell-ignite-2022-announcements/ba-p/3649324

  • general availability of Azure PowerShell modules version 9
  • added 12 modules supporting new services and added more than 500 cmdlets
  • With Az 9 we are providing an actionable error message that indicates why a cmdlet is not found.
  • With Az Config you can CENTRALLY CONFIGURE Azure PowerShell settings

Active Directory Connector (ADC) for Arc-enabled SQL Managed Instance is now generally available!

https://techcommunity.microsoft.com/t5/azure-arc-blog/active-directory-connector-adc-for-arc-enabled-sql-managed/ba-p/3652020

Azure Arc-enabled data services support Active Directory (AD) for Identity and Access Management (IAM). The Arc-enabled SQL Managed Instance uses an existing on-premises Active Directory (AD) domain for authentication.

Azure Backup

Public preview: Immutable vaults for Azure Backup

https://azure.microsoft.com/en-gb/updates/azure-backup-immutable-vaults-preview/

With immutable vaults, Azure Backup provides you an option to ensure that recovery points that are once created cannot be deleted before their intended expiry time.

Public preview: Multi-user authorization for Backup vaults

https://azure.microsoft.com/en-gb/updates/azure-backup-mua-backup-vaults-preview/

Multi-user authorization (MUA) for Backup adds an additional layer of protection for critical operations on your Backup vaults, providing greater security for your backups. To provide multi-user authorization, Backup uses a resource guard to ensure critical operations are performed with proper authorization

Public preview: Enhanced soft delete for Azure Backup

http://azure.microsoft.com/en-us/updates/azure-backup-enhanced-soft-delete-preview/

With enhanced soft delete, you get the ability to make soft delete irreversible, which protects soft delete from being disabled by any malicious actors. Hence, enhanced soft delete provides better protection for your backups against various threats. With enhanced soft delete, you get the ability to make soft delete irreversible, which protects soft delete from being disabled by any malicious actors. Hence, enhanced soft delete provides better protection for your backups against various threats.

General availability: Zone-redundant storage support by Azure Backup

http://azure.microsoft.com/en-us/updates/azurebackupzrssupport/

With the general availability of this feature, you have a broader set of redundancy or storage replication options to choose from for your backup data. Based on your data residency, data resiliency and total cost of ownership (TCO) requirements, you can select either locally redundant storage (LRS), zone-redundant storage (ZRS) or geo-redundant storage (GRS).

After Ignite – Up To November 30th

Cost Management

General availability: Azure savings plan for compute

https://azure.microsoft.com/en-gb/updates/general-availability-azure-savings-plan-for-compute/

The savings plan unlocks lower prices on select compute services when customers commit to spend a fixed hourly amount for one or three years. Choose whether to pay all up front or monthly at no extra cost.

General availability: Virtual Machine software reservations

https://azure.microsoft.com/en-gb/updates/general-availability-virtual-machine-software-reservations/

You can now save on Virtual Machine software from third-party publishers by purchasing software reservations.

Hybrid

Generally available: Auto Extension upgrade for Arc enabled Servers

https://azure.microsoft.com/en-us/updates/auto-extension-upgrade-for-arc-servers/

Automatic Extension upgrade is now generally available for Arc enabled Servers using eligible VM extensions. With this release we are adding support for Azure Portal, PowerShell, CLI, and automatic rollback of failed upgrades

Networking

Visualize and monitor Azure & hybrid networks with Azure Network Watcher

http://azure.microsoft.com/blog/visualize-and-monitor-azure-hybrid-networks-with-azure-network-watcher/

Azure Network Watcher provides an entire suite of tools to visualize, monitor, diagnose, and troubleshoot network issues across Azure and Hybrid cloud environments.

Azure Virtual WAN simplifies networking needs

http://azure.microsoft.com/blog/networking-needs-simplified-with-azure-virtual-wan/

  • Multipool user group support preview
  • Secure hub routing intent preview
  • Hub routing preference (HRP) is generally available
  • Bypass next hop IP for workloads within a spoke VNet connected to the virtual WAN hub generally available
  • Border Gateway Protocol (BGP) Peering with a virtual hub is generally available
  • BGP dashboard is now generally available
  • Virtual Network Gateway VPN over ExpressRoute private peering (AZ and non-AZ regions) is generally available
  • Custom traffic selectors (portal)–generally available
  • High availability for Azure VPN client using secondary profile is generally available
  • ExpressRoute circuit with visibility of Virtual WAN connection
  • Fortinet SDWAN is generally available
  • Aruba EdgeConnect Enterprise SDWAN preview
  • Checkpoint NG Firewall preview

Generally available: Block domain fronting behavior on newly created customer resources

https://azure.microsoft.com/en-us/updates/generally-available-block-domain-fronting-behavior-on-newly-created-customer-resources/

beginning November 8, 2022, all newly created Azure Front Door, Azure Front Door (classic) or Azure CDN Standard from Microsoft (classic) resources will block any HTTP request that exhibits domain fronting behavior.

General availability: Default Rule Set 2.1 for Azure Web Application Firewall

https://azure.microsoft.com/en-gb/updates/general-availability-default-rule-set-21-for-azure-web-application-firewall/

Increase your security posture and reduce false positives with Default Rule Set 2.1, now generally available on Azure’s global Web Application Firewall running on Azure Front Door.

Evolving networking with a DPU-powered edge

https://techcommunity.microsoft.com/t5/azure-stack-blog/evolving-networking-with-a-dpu-powered-edge/ba-p/3672898

SmartNICs or Data Processing Units (DPUs) bring an opportunity to double down on the benefits of a software-defined infrastructure without sacrificing the host resources needed by your line-of-business apps in your (virtual machines) VMs or containers. With a DPU, we can enable SR-IOV usage removing the host CPU consumption incurred by the synthetic datapath, alongside the SDN benefits.

Public preview: Azure Front Door zero downtime migration

http://azure.microsoft.com/en-us/updates/public-preview-azure-front-door-zero-downtime-migration/

You can use this feature to migrate Azure Front Door (classic) to Azure Front Door Standard and Premium with zero downtime.

Public preview: Azure Front Door integration with managed identities

http://azure.microsoft.com/en-us/updates/public-preview-azure-front-door-integration-with-managed-identities/

Azure Front Door Standard and Premium supports enabling managed identities for Azure Front Door to access Azure Key Vault.

Public preview: Upgrade from Azure Front Door Standard to Premium tier

https://azure.microsoft.com/en-gb/updates/public-preview-upgrade-from-azure-front-door-standard-to-premium-tier/

You can now use this feature to upgrade your Azure Front Door Standard profile to Premium tier without downtime.

General availability: Per Rule Actions on regional Web Application Firewall

https://azure.microsoft.com/en-us/updates/general-availability-per-rule-actions-on-regional-web-application-firewall/

Azure’s regional Web Application Firewall (WAF) with Application Gateway running the Bot Protection rule set and Core Rule Set (CRS) 3.2 or higher now supports setting actions on a rule-by-rule basis.

General availability: TLS 1.3 with Application Gateway

http://azure.microsoft.com/en-us/updates/tls1-3-application-gateway-ga/

Start using the new policies with TLS 1.3 for your Azure Application Gateway to improve security and performance.

Announcing new capabilities for Azure Firewall

http://azure.microsoft.com/blog/announcing-new-capabilities-for-azure-firewall/

  • New GA regions in Qatar central, China East, and China North
  • IDPS Private IP ranges now generally available.
  • Single Click Upgrade/Downgrade now in preview.
  • Enhanced Threat Intelligence now in preview.
  • KeyVault with zero internet exposure now in preview.

AKS

Dapr v1.9.0 now available in the Dapr extension for AKS and Arc-enabled Kubernetes

https://techcommunity.microsoft.com/t5/azure-developer-community-blog/dapr-v1-9-0-now-available-in-the-dapr-extension-for-aks-and-arc/ba-p/3655958

The Dapr v1.9.0 release offers several new features, including pluggable components, resiliency metrics, and app health checks, as well as many fixes in the core runtime and components.

Generally available: Premium SSD v2 disks available on Azure Disk CSI driver

https://azure.microsoft.com/en-gb/updates/generally-available-premium-ssd-v2-disks-available-on-azure-disk-csi-driver/

Premium SSD v2 support is now generally available on AKS.

Public preview: AKS image cleaner

https://azure.microsoft.com/en-gb/updates/public-preview-aks-image-cleaner/

You can now more easily remove unused and vulnerable images stored on AKS nodes.

Public preview: IPVS load balancer support in AKS

https://azure.microsoft.com/en-gb/updates/public-preview-ipvs-load-balancer-support-in-aks/

You can now use the IP Virtual Server (IPVS) load balancer with AKS, with configurable connection scheduling and TCP/UDP timeouts.

Public preview: Azure CNI Powered by Cilium

https://azure.microsoft.com/en-gb/updates/public-preview-azure-cni-powered-by-cilium/

Leverage next generation eBPF dataplane for pod networking, Kubernetes network policies and service load balancing.

Public preview: Rotate SSH keys on existing AKS nodepools

http://azure.microsoft.com/en-us/updates/public-preview-rotate-ssh-keys-on-existing-aks-nodepools/

You can now update SSH keys on existing AKS nodepools post deployment.

Azure VMware Solution

Generally available: New node sizing for Azure VMware Solution

https://azure.microsoft.com/en-gb/updates/generally-available-new-node-sizing-for-azure-vmware-solution/

Optimize workloads with new node sizes, AV52, and AV36P, now generally available in Azure VMware Solution.

Generally available: Azure NetApp Files datastores for Azure VMware Solution

http://azure.microsoft.com/en-us/updates/generally-available-azure-netapp-files-datastores-for-azure-vmware-solution/

Azure NetApp Files datastores is now generally available to run your storage intensive workloads on Azure VMware Solution (AVS).

Virtual Machines

General availability: Ephemeral OS disk support for confidential virtual machines

https://azure.microsoft.com/en-gb/updates/general-availability-ephemeral-os-disk-support-for-confidential-virtual-machines/

Create confidential VMs using Ephemeral OS disks for your stateless workloads.

General availability: New cost recommendations for Virtual Machine Scale Sets

https://azure.microsoft.com/en-gb/updates/general-availability-new-cost-recommendations-for-virtual-machine-scale-sets/

Azure Advisor has expanded recommendations to include cost optimisation recommendation for Virtual Machine Scale Sets too.

Microsoft Intune user scope configuration for Azure Virtual Desktop multi-session VMs is now GA

https://techcommunity.microsoft.com/t5/azure-virtual-desktop-blog/microsoft-intune-user-scope-configuration-for-azure-virtual/ba-p/3667410

This new update enables you to configure user scope policies using settings catalog, configure user certificates, and configure PowerShell scripts in user context.

Generally available: Encrypt managed disks with cross-tenant customer-managed keys

http://azure.microsoft.com/en-us/updates/generally-available-encrypt-managed-disks-with-crosstenant-customermanaged-keys/

Many service providers building Software as a Service (SaaS) offerings on Azure want to give their customers the option of managing their own encryption keys.

General availability: Bot Manager Rule Set 1.0 on regional Web Application Firewall

http://azure.microsoft.com/en-us/updates/general-availability-bot-manager-rule-set-10-on-regional-azure-web-application-firewall/

This rule set provides you enhanced protection against bots and provides granular control over bots detected by WAF by categorizing bot traffic as good, bad, or unknown bots.

Public preview: Azure Bastion now support shareable links

http://azure.microsoft.com/en-us/updates/azure-bastion-shareable-links/

Shareable links allows users to connect to target resources via Azure Bastion without access to the Azure portal.

Storage

Generally available: SFTP support for Azure Blob Storage

http://azure.microsoft.com/en-us/updates/sftp-support-for-azure-blob-storage-now-generally-available/

Azure Blob Storage now supports provisioning an SFTP endpoint with just one click.

Public preview: Availability zone volume placement for Azure NetApp Files

http://azure.microsoft.com/en-us/updates/public-preview-availability-zone-volume-placement-for-azure-netapp-files/

Deploy new Azure NetApp Files volumes in Azure availability zones (AZs) of your choice to support workloads across multiple availability zones.

App Services

App Service Environment version 1 and version 2 will be retired on 31 August 2024

http://azure.microsoft.com/en-us/updates/app-service-environment-version-1-and-version-2-will-be-retired-on-31-august-2024/

Migrate to App Service Environment version 3 by 31 August 2024

Generally available: Azure Static Web Apps now fully supports .NET 7

https://azure.microsoft.com/en-gb/updates/generally-available-azure-static-web-apps-now-fully-supports-net-7/

Azure Static Web Apps now supports building and deploying full-stack .NET 7.0 isolated applications.

Public preview: Azure Static Web Apps now Supports Node 18

https://azure.microsoft.com/en-gb/updates/public-preview-azure-static-web-apps-now-supports-node-18/

Azure Static Web Apps now supports building and deploying full-stack Node 18 applications.

Generally available: Static Web Apps support for skipping API builds

https://azure.microsoft.com/en-gb/updates/generally-available-static-web-apps-support-for-skipping-api-builds/

Azure Static Web Apps provides the option to skip the default API builds via GitHub Actions and Azure pipelines. While setting up the YAML build configuration, you can set the skip_api_build flag to true in order to skip building the APIs.

Generally available: Static Web Apps support for stable URLs for preview environments

https://azure.microsoft.com/en-gb/updates/generally-available-static-web-apps-support-for-stable-urls-for-preview-environments/

Use stable URLs with Azure Static Web Apps preview environments.

Generally available: Static Web Apps support for Gitlab and Bitbucket

https://azure.microsoft.com/en-gb/updates/generally-available-static-web-apps-support-for-gitlab-and-bitbucket/

Deploy Static Web Apps using Gitlab and Bitbucket as CI/CD providers.

Generally available: Static Web Apps support for preview environments in Azure DevOps

https://azure.microsoft.com/en-gb/updates/generally-available-static-web-apps-support-for-preview-environments-in-azure-devops/

Deploy applications to staging environments using Azure DevOps.

Public preview: Go language support on Azure App Service

http://azure.microsoft.com/en-us/updates/public-preview-go-language-support-on-azure-app-service/

Go language (v1.18 and v1.19) is natively supported on Azure App Service, helping developers innovate faster using the best fully managed app platform for cloud-centric web apps. The language support is available as an experimental language release on Linux App Service in November 2022.

Generally available Day 0 support for .NET 7.0 on App Service

https://azure.microsoft.com/en-gb/updates/generally-available-day-0-support-for-net-70-on-app-service/

developers are immediately unblocked to try, test, and deploy .NET apps targeting the version of .NET accelerating time-to-market on the platform they know and use today. It is expected to be available in Q2 FY23.

Miscellaneous

Secure your digital payment system in the cloud with Azure Payment HSM—now generally available

http://azure.microsoft.com/blog/secure-your-digital-payment-system-in-the-cloud-with-azure-payment-hsm-now-generally-available/

the general availability of Azure Payment HSM, a BareMetal Infrastructure as a service (IaaS) that enables customers to have native access to payment HSM in the Azure cloud. With Azure Payment HSM, customers can seamlessly migrate PCI workloads to Azure and meet the most stringent security, audit compliance, low latency, and high-performance requirements needed by the Payment Card Industry (PCI).

Automated Key Rotation Generally Available on Azure Key Vault Managed HSM

https://techcommunity.microsoft.com/t5/azure-confidential-computing/automated-key-rotation-generally-available-on-azure-key-vault/ba-p/3671635

The feature allows you to set up an auto-rotation policy that automatically generates a new key version of the customer-managed key (CMK) stored in the HSM at a specified frequency.

General availability: Azure Automation supports Availability zones

http://azure.microsoft.com/en-us/updates/azure-automation-availability-zones/

Azure Automation now supports Availability zones to provide improved resiliency and reliability to the service, runbooks and other automation assets.

Public preview: Microsoft Azure Managed HSM TLS Offload Library

https://azure.microsoft.com/en-gb/updates/public-preview-managed-hsm-tls-offload-library/

Azure Managed HSM now supports SSL/TLS Offload for F5 and Nginx.

Generally available: Additional Always Free Services for Azure Free Account and PAYG

http://azure.microsoft.com/en-us/updates/generally-available-additional-always-free-services-for-azure-free-account-and-payg/

With an Azure free account, you can explore with free amounts of 55+ always free services.

AVD

Announcing general availability of FSLogix profiles for Azure AD-joined VMs in Azure Virtual Desktop

https://techcommunity.microsoft.com/t5/azure-virtual-desktop-blog/announcing-general-availability-of-fslogix-profiles-for-azure-ad/ba-p/3671310

By leveraging Azure AD Kerberos with Azure Files, you can seamlessly access file shares from Azure AD-joined VMs and use them to store your FSLogix profile containers.

Management

General availability: Manage your Log Analytics Tables in Azure Portal

http://azure.microsoft.com/en-us/updates/general-availability-manage-your-log-analytics-tables-in-azure-portal/

announcing the general availability of a new experience for managing Azure Log Analytics table metadata from the Azure Portal. With this new UI you can view and edit table properties directly from Azure Portal in Log Analytics workspaces experience.

New Project Flash Update: Advancing Azure Virtual Machine availability monitoring

http://azure.microsoft.com/blog/advancing-azure-virtual-machine-availability-monitoring-with-project-flash-update/

  • General availability of VM availability information in Azure Resource Graph
  • Preview of a VM availability metric in Azure Monitor
  • Preview of VM availability status change events via Azure Event Grid

General availability: Azure Monitor agent custom and IIS logs

http://azure.microsoft.com/en-us/updates/general-availability-azure-monitor-agent-custom-and-iis-logs/

This new capability is designed to enable customers to collect their text-based logs generated in their service or application. Likewise, Internet Information Service (IIS) logs for a customers’ service can be collected and transferred into a Log Analytics Workspace table for analysis.

General availability: Azure Monitor Logs, custom log API and ingestion-time transformations

http://azure.microsoft.com/en-us/updates/general-availability-azure-monitor-logs-custom-log-api-and-ingestiontime-transformations/

With these new features, you will be able to add a custom ingestion-time transformation to data following into Azure Monitor Logs. These transformations can be used to set up ingestion-time extraction of fields and parsing of complex logs, obfuscation of sensitive data, removal of unneeded fields or even dropping full events for cost control, and many more advanced possibilities.

Announcing GA of revamped Custom Logs features

https://techcommunity.microsoft.com/t5/azure-observability-blog/announcing-ga-of-revamped-custom-logs-features/ba-p/3687366

  • GA of the Log Ingestion API
  • GA of the Ingestion-time Transformations feature
  • A nominal fee per GB dropped will be charged for any data dropped beyond 50% of incoming data, calculated daily

Azure Backup

Limited preview: Azure Backup support for confidential VMs using Platform Managed Keys

https://azure.microsoft.com/en-gb/updates/limited-preview-azure-backup-support-for-confidential-virtual-machines-using-platform-managed-keys/

You can use this feature to back up confidential VMs using Platform Managed Keys.

Public preview: Cross Subscription Restore for Azure Virtual Machines

http://azure.microsoft.com/en-us/updates/preview-cross-subcription-restore-for-azure-virtual-machines/

Cross Subscription Restore allows you to restore Azure Virtual Machine, through create new or restore disks, to any subscription (honoring the RBAC capabilities) from the restore point created by Azure Backup.

Azure Firewall Basic – For Small/Medium Business & “Branch”

Microsoft has just announced a lower cost SKU of Azure Firewall, Basic, that is aimed at small/medium business but could also play a role in “branch office” deployments in Microsoft Azure.

Standard & Premium

Azure Firewall launched with a Standard SKU several years ago. The Standard SKU offered a lot of features, but some things deemed necessary for security were missing: IDPS and TLS Inspection were top of the list. Microsoft added a Premium SKU that added those features as well as fuller web category inspection and URL filtering (not just FQDN).

However, some customers didn’t adopt Azure Firewall because of the price. A lot of those customers were small-medium businesses (SMBs). Another scenario that might be affected is a “branch office” in an Azure region – a smaller footprint that is closer to clients that isn’t a main deployment.

Launching The Basic SKU

Microsoft has been working on a lower cost SKU for quite a while. The biggest challenge, I think, that they faced was trying to figure out how to balance feature, performance, and availability with price. They know that the target market has a finite budget, but there are necessary feature requirements. Every customer is different, so I guess when face with this conundrum, one needs to satisfy the needs of 80% of customers.

The clues for a new SKU have been publicly visible for quite a while – the ARM reference for Azure Firewall documented that a Basic SKU existed somewhere in Azure (in private preview). Tonight, Microsoft launched the Basic SKU inpublic Preview. A longer blog post adds some details.

Introducing the Azure Firewall

The primary target market for the Basic SKU hasn’t deployed a firewall appliance of any kind in Azure – if they are in Azure then they are most likely only using NSGs for security – which operates only at the transport protocol (TCP, UDP, ICMP) layer in a decentralised way.

The Azure Firewall is a firewall appliance, allowing centralised control. It should be deployed with NSGs and resource firewalls for layered protection, and where there is a zero-trust configuration (deny all by default) in all directions, even inside of a workload.

The Azure Firewall is native to Microsoft Azure – you don’t need a third party license or support contract. It is fully deployable and configured as code (ARM, Bicep, Terraform, Pulumi, etc), making it ideal for DevSecOps. Azure Firewall is much easier to learn than NVAs because the firewall is easily available through an Azure subscription and the training (Microsoft Learn) is publicly available – not hidden behind classic training paywalls. Thanks to the community and a platform model, I expect that more people are learning Azure Firewall than any other kind of firewall today – skills are in short supply so using native tech that is easy to learn and many are learning just makes sense.

Comparing Azure Basic With Standard and Premium

Microsoft helpfully put together a table to compare the 3 SKUs:

Comparing Azure Firewall Basic with Standard and Premium

Another difference with the Basic SKU is that you must deploy the AzureFirewallManagementSubnet in addition to the AzureFirewallSubnet – this additional subnet is often associated with forced tunneling. The result is that the firewall will have a second public IP address that is used only for management tasks.

Pricing

The Basic SKU follows the same price model as the higher SKUs: a base compute cost and a data processing cost. The shared pricing is for the Preview so it is subject to change.

The Basic SKU base compute (deployment) cost is €300.03 per month in West Europe. That’s less than 1/3 of the cost of the Standard SKU at €947.54 per month. The data processing cost for the Basic SKU is higher at €0.068 per GB. However, the amount of data passing through such a firewall deployment will be much lower so it probably will not be a huge add-on.

Preview Deployment Error

At this time, the Basic SKU is in preview. You must enable the preview in your subscription. If you do not do this, your deployment will fail with this error:

“code”: “FirewallPolicyMissingRequiredFeatureAllowBasic”,

“message”: “Subscription ‘someGuid’ is missing required feature ‘Microsoft.Network/AzureFirewallBasic’ for Basic policies.”

Some Interesting Notes

I’ve not had a chance to do much work with the Basic SKU – work is pretty crazy lately. But here are two things to note:

  • A hub & spoke deployment is still recommended, even for SMBs.
  • Availability zones are supported for higher availability.
  • You are forced to use Azure Firewall Manager/Azure Firewall Policy – this is a good thing because newer features are only in the new management plane.

Final Thoughts

The new SKU of Azure Firewall should add new customers to this service. I also expect that larger enterprises will also be interested – not every deployment needs the full blown Standard/Premium deployment but some form of firewall is still required.

3 Tips For MS Certification Hands-On-Labs

Here are my three tips for improving your results in a hands-on-lab in Microsoft certification exam:

  1. Read the task description carefully.
  2. Optimise your time.
  3. Review your work.

About Hands-On-Labs

If you’ve sat a Microsoft century exam over the past quarter century then you are familiar with the traditional format:

  • Either a simple scenario or a case study.
  • Multiple choice questions where you select one best answer or multiple answers that are correct or part of the best solution, or sometimes ordering the steps.
  • “Correct” answers that are wrong and “wrong” answers that are right depending on feature/update releases and when the question was probably written.
  • Trick questions that are quite unfair.

I sat the AZ-700: Designing and Implementing Microsoft Azure Networking Solutions this week and was surprised to see a hands-on-lab at the very end. Before the lab appeared I had approximately an hour left in the exam. When I was finished, I had 5 minutes left. The exercises were not hard, especially for anyone used to deploying Azure networking resources, but it was time consuming and there were a lot of tasks to complete.

The lab appeared at the end of the exam. It provided a username and password. The instructions do not offer a copy/paste feature in the Azure Portal which is embedded in the test. But you can click the username and password to get them to appear in the log-in screen. The absence of copy/paste means that you need to be careful when you are asked to enter a specific name for a resource.

The lab was made up of a number of exercises. Each exercise was discrete in my exam – no exercise depended on another. Each exercise had a description of differing complexity and clarity. Some were precise and some were vague. Some were short tasks and others were long-running tasks.

I found myself answering a comment on LinkedIn this morning and thought “this would be a nice blog post”. So on with the tips!

Read The Task Description Carefully

Just like the multiple-choice questions, the exercises are probably (I’m guessing) written by Microsoft Certified Trainers (MCTs) who may or may not be experts in the exam content – this sort of comment often makes MCTs angry and defensive. It’s clear from the language that some of the authors don’t have a clue – this is where MCTs say “leave feedback” but I would have spent another 2 hours leaving feedback when I had a family to get home to!

A task might have multiple ways to complete it. I can’t share specifics from the exam, but one of the tasks was very vague and offered no context. Without thinking too much I thought immediately of 3 possible deployments that could solve the issue. Which was right? Were all 3 right? I guessed with the one that would require the least work – it felt right based on some of the language but I was reading between lines and trying to think what the author was thinking.

Read the task instructions carefully. Take notes of things to complete – use the dry marker pens and boards you are given and check steps off as you do them. If a required resource is named, then note the case and duplicate that in your lab – Azure might drop it to lower case because that’s what it might do for that resource type!

Don’t jump to any assumptions. Look for clue words that hunt at requirements or things to avoid. In the unclear question that I had, there was one word that lead me to choose the approach that I took. I’ve no idea if the answer was right or wrong, but it felt right.

  1. Read the task carefully.
  2. Take notes on actions to create a checklist.
  3. Re-read the task looking for clue words.
  4. Verify your checklist is complete.
  5. Tick off items on the checklist as you work.

Optimise Your Time

I had 12 tasks (I think) to complete after answering dozens of multiple choice questions. I had an hour or so left in the exam and that hour flew by. As I said earlier, some tasks were quick. Some tasks required a lot of work. And some tasks were long-running.

In my exam, the tasks were independent of each other. That meant I could start on task 2 while a resource for task 1 was still deploying.

The Azure Portal can offer several ways to accomplish a task. You can build out each resource you require in individual wizards. Sometimes, the last “big” resource that you need has a wizard that can deploy everything – that’s the method that you want. Practice is your friend here, especially if you normally work using infrastructure-as-code (like me) and rarely deploy in the Azure Portal. Find different ways to deploy things and compare which is more efficient for basic deployments.

There are certain resources in Azure networking that take 10-45 minutes. If you have a task such as this then do not wait for the deployment to complete. Jump ahead to the next task and start reading.

You might find yourself working on 2-3 tasks at once if you use this approach. This is where tracking your work becomes critical. Earlier, I stated that you should track the requirements of a task using a checklist. You should also track the completion status of each task – you don’t want to forget to complete a task where you are waiting on a resource to deploy. Each task has a “Mark As Complete” button – use it and don’t consider the lab as complete until all tasks have green check marks.

  1. Practice deployments in the Azure Portal.
  2. Choose the deployment method that will complete more of the task requests in less time.
  3. Do not wait on long-running deployments.
  4. Track task completion using the “Mark As Complete” button.

Review Your Work

In my exam, the tasks normally did not instruct you to use a resource name. So I created names using the naming standard that I am used to. When I had completed all the tasks – all had green check marks – I decided to review my work. I read through the task requirements again and verified the results in the Azure Portal. I found that one task asked me to create a resource with a specific name and I had created it using my normal naming standard. I fixed my error and continued to check everything.

When you have finished your work, go through the exercise descriptions again. Confirm that the checklists are complete. And compare the asks with what you have done in the Azure Portal to verify that everything is done as it is required.

  1. Read the task descriptions again.
  2. Compare with your checklist.
  3. Compare with the results in the Azure Portal.

My Experience

It took me a few minutes to get over the shock of doing a hands-on-lab. I dreaded every minute because I have heard the horror tales of labs being slow or crashing mid-exam. I also was glad of blogging and lab work – in my day job I rarely use the Azure Portal to deploy networking (or any) resources.

But once settled in, I found that the labs were not difficult. The asks were not complicated – they were a mixture of vague and detailed:

  • How to decide what to deploy wasn’t written down.
  • Only one task had a requirement to name 1 resource.

The thing that really shocked me, though, was that I did not learn my result at the end of the exam. The normal Microsoft exam experience is that you confirm that you are finished the exam, you spend a few painful seconds wondering if the program has crashed, and a result appears. Instead, I was told to log into http://www.microsoft.com/learning/dashboard later in the day to see my result! I had to drive home and then take care of my kids so it was 90 minutes later when I finally got to sign in and navigate to see my result – which was a pass with no score shared. So I guess that I did OK in the labs.

Dealing With Azure Capacity Shortages

In this post, I will discuss the recent news that Azure is having some serious capacity issues affecting customers in 2022 that could last into 2023.

The News

Personally, I didn’t think that Azure having capacity issues was news. I thought that thanks to all the jokes on social media, everyone took it for granted that the effects of the pandemic and cargo crises had crippled electronics supplies the world over. Even the recent news that the annual MVP renewal (July 1st) was being postponed to July 5 was jokingly blamed on the capacity issues.

Last Friday, The Information (subscription required) reported:

… more than two dozen Azure data centers in countries around the world are operating with limited server capacity available to customers, according to two current Microsoft managers contending with the issue and an engineer who works for a major customer.

This is not the news we want to hear when there are repeated reports that cloud adoption is increasing, Azure adoption is growing faster than AWS adoption, and Microsoft’s total (including SaaS) cloud business is (marginally) the largest in the industry.

The Effect

Most people who have used Azure for a few years have seen the dreaded deployment fails – you cannot get capacity because the region you have selected doesn’t have it. And the “helpful” support agent tells you to try a less-impacted region on another continent.

That suggestion indicates how little that person cares or the lack of understanding about IT performance or compliance issues.

I work mostly with Norwegian clients. Typically, they need to stay inside the EU (using West Europe, the largest hero region), and often they have to stay inside Norway (Norway East, because Norway West is not publicly available to customers). If one of those clients tries to start something or create something in their chosen region then being told to deploy in Asia or the US isn’t going to work. If they chose Norway East because they need to comply with the local “Arkiv” (archive) law then venturing outside the border is not an option.

Let’s say you’ve built a highly integrated system in West US. You have lots of containers and functions, all integrating with each other, and hitting databases. Your workload is time-sensitive so you need to minimise every millisecond of latency. Heck, you have even started to use VMs to get access to proximity placement groups to control physical distances between tiers of your workloads. And then a support engineer tells you that your additional scale-out capacity should be halfway around the planet? Seriously?

Solutions

Other than some miracle where manufacturing backlogs and shipping mismanagement are solved, you will need to minimise risks of this sort of thing happening to you.

Auto-Scaling

Customers use auto-scaling to reduce costs. You dynamically create or power-up (allocate to a running/billing state) compute instances to stay slightly ahead of demand. And when demand (profits) reduce, you dynamically destroy or power down (deallocate to a stopped/non-billing state) unneeded compute instances.

When capacity is not available, auto-scaling can be a gamble. You want to minimise wastage during non-peak periods, but not having enough compute instances could damage revenue/operations more than having unused compute instances. For example, your Citrix/Azure Virtual Desktop instances are required for people to work, but when people try to log in on Monday morning, Azure cannot supply enough machines to scale-out your worker pools. Whoops! You saved some money over the weekend, but now only a small percentage of your employees can work.

In these times of shortage, you need to hold on to what you have got. It is worth considering the disablement of auto-scaling and going back to old practices of trying to figure out what you need, and keeping that capacity running.

Virtual Machine Migration

After the summer, I will have some involvement in three migration projects. Two of those could result in large numbers of virtual machines being deployed to Azure. All three projects have hard deadlines for completion. The last thing that those clients are going to want to hear is: “Sorry, your business plans are not feasible because there aren’t enough servers in the Cloud that IT has selected”.

Microsoft offers a program where you can reserve virtual machine capacity for selected series of virtual machines, called On-demand Capacity Reservation. In short, you pay for the price of the machine you want, before you deploy it, to guarantee the capacity that you want when you need it later.

Or as some cynics might say: you pay a vig to Microsoft for capacity that you should have expected in a hyper-scale cloud in the first place.

If you are in a scenario where you have an upcoming spike in capacity, such as a migration, and you need to be sure the capacity will be there, then this form of reservation must be strongly considered. However, telling a client that they need to pay for compute weeks before they need it – will be a hard sell.

SKU Choice

When we deploy new tech, we always want the latest and greatest. Why would you deploy a years-old D_v3 when there’s a D_v5 with a faster processor? If you really need the performance of the newer SKU, then I get the choice. But how many workloads genuinely need that kind of sizing. I’ve rarely encountered one.

Under the covers of the platform, that v5 is limited to new hardware with matching physical processors. That is the same capacity that Microsoft (and others) cannot get their hands on. However, the v3 is able to run on the v3 hardware, and thanks to Hyper-V processor management features, it is also able to run on newer hardware. So if you want more potential host capacity to run your compute instances, then you will choose older SKUs that run on older processors.

This advice will apply to any resource type – not just virtual machines. In the end, everything (including so-called serverless computing) is a virtual machine under the platform, except for items such as VMware hosts and SAP Hana machines which are physical servers.

Summary

It sounds like we are facing some issues. Azure might be making the headlines now, but Microsoft uses the same physical components as everyone else – there is only AMD and Intel, pretty much everything electronic is assembled in China, and everything needs months to move by cargo ship. We are going to have to box clever to ensure that we have enough capacity for resources such as virtual machines, app services, containers, databases, and so on – everything that relies on compute instances under the covers.

Deploy Shared Azure Bastion To Virtual WAN Spoke

In this post, I will explain how you can deploy Azure Bastion into a spoke in a hub & spoke architecture with a Virtual WAN hub – and use that Bastion to securely log into virtual machines in other spokes using RDP or SSH. I will also explain why this has limitations in a hub & spoke architecture with a VNet-based hub.

The Need

Even organisations that opt for a PaaS-only Azure implementation need virtual machines. Once you do network security, you need virtual machines for those DevOps build agents or those GitHub runners. And realistically, migrated legacy workloads need VMs. And things like AKS and HPC are based on VMs that you build and troubleshoot. So you need VMs. And therefore, you need a secure way to log into those VMs.

Those of us who want an air gap between the PC and the servers have tried things like RD Gateway and Guacamole. Neither is perfect. Ideally, we want Azure AD integration (for Premium security features) and a platform resource (to minimise maintenance).

And along came Azure Bastion. At first reading, it seemed ideal. And then we started to discover warts. Many of those warts were cleaned up. Bastion got support for a desktop client through a CLI login. A hub deployment was possible – if you use a VNet-based hub – but it gave Bastion users (including external support) staff a map of your entire Azure network because they read access to the hub VNet – and all its peering connections. For many of us, that left us with deploying Bastion in every subnet – both costly and a waste of IP space.

We needed an Azure Bastion that we could deploy once in a spoke. We could log into it, route through the firewall in the hub, and log into VMs running in other spokes.

IP-Based Connection

Microsoft announced a new feature in the Standard tier of Azure Bastion called IP-Based Connection. With this feature, you can log into a virtual machine across an IP network from your Bastion. That means you can log into:

  • Virtual machines in the same subnet or virtual network
  • Virtual machines in other Azure virtual networks
  • On-premises computers across site-to-site network connections such as VPN or ExpressRoute

The assumptions are that:

  • The NSG protecting the Azure virtual machine allows SSH/RDP from AzureBastionSubnet o the virtual machine.
  • The hub firewall (if you have one) allows SSH/RDP from AzureBastionSubnet to the virtual machine.
  • The on-premises firewall, if the virtual machine is on-premises, allows RDP/SSH from AzureBastionSubnet to the computer.
  • The OS of the virtual machine or computer allows RDP/SSH rom AzureBastionSubnet.

VNet-Based Hub

In my first experiment, I tried deploying a shared Azure Bastion in a spoke with a VNet-based hub. I could deploy it no problem but Azure Bastion could not route to other spokes sharing the hub. Why?

There were no routes from the spoke subnets to other spoke subnets. But that’s OK – I know how to fix that.

Let’s say my entire Azure network is in the 10.1.0.0/16 address space. Well, if I want to route to any spoke in that address space, I can:

  1. Create a route table and *cough* associate it with the AzureBastionSubnet
  2. Add a user-defined route (UDR) for 10.1.0.0/16 with a next hop of the hub firewall (or a routing appliance).

Azure Bastion needs to have a route of 0.0.0.0/0 to Internet so you can’t do the usual spoke thing with the UDR so I could leave the default Internet route in place.

Did it work? No – that’s because AzureBastionSubnet has a hard-coded rule to prevent the association of a route table. I guess that Microsoft had too many support calls with people doing bad things with a route table associated with AzureBastionSubnet.

It turns out that the only way to get a route from one spoke to another with a VNet-based hub is:

  1. Use Azure Virtual Network Manager (AVNM – currently in preview) to peer your spokes with transitive peering.
  2. Do not expect spoke-to-spoke traffic to flow through a hub firewall – AVNM does not support configuring a next-hop.

That means the only shared Azure Bastion option for VNet-based hubs is to deploy Bastion in the hub and leave all your peering connections visible. Ick!

vWAN-Based Hub

I really like using Azure Virtual WAN (vWAN) for my hub and spoke – and almost none of my customers use it for SD-WAN (the primary use case). The reasons I like it are:

  • It pushes the hub into the platform, reducing administrative efforts
  • Routing becomes something that you push from the hub using eBGP

“Ah – what’s that you say about eBGP, Aidan?”

You can create route tables in Virtual WAN Hubs – let’s call them hub route tables. Then in the properties of a spoke virtual network connection you can configure:

  • Propagation: Have the route table learn routes from the spoke virtual network using eBGP.
  • Association: Share routes from the route table to the spoke virtual network.

And you can put static routes into a hub route table.

My Scenario

A shared Azure Bastion in the spoke of an Azure Virtual WAN hub

When I deploy a Virtual WAN Hub, I choose the Secured Virtual WAN Hub option. This places an Azure Firewall in the hub. I then add a static route for the 0.0.0.0/0 and private IP address spaces to route via the Azure Firewall in the build-in Default hub route table.

All spokes:

  • Propagate to the built-in None hub route table, so the routes of the spoke are forgotten.
  • Associate with the built-in Default hub route table, so they learn the next hop to 0.0.0.0/0 and the private IP address spaces is via the firewall.

I can deploy Azure Bastion into a spoke but this spoke will require a different route configuration. That is because I use a firewall to isolate the spokes. If I had open spoke-to-spoke traffic then Bastion would probably just work. My scenario is actually simple to fix:

  1. Create a new hub route table, maybe called Bastion.
  2. Add a static route to the rest of the hub and spoke (10.1.0.0/16 in my example) with a next-hop of the hub firewall.
  3. Configure the Bastion spoke connection to associate with the Bastion hub route table and propagate to none.

Now:

  • The Bastion will use the firewall as the next hop to all other spokes.
  • Go directly to Internet for the control plane traffic, using the default route in the subnet.
  • Other spokes will have the same route back to the Bastion using the firewall as the next-hop.

Finally, you need to ensure that Firewall and NSG rules allow RDP/SSH from AzureBastionSubnet in the Bastion spoke to the VMs in other spokes. And it works! All an operator/developer/support staff member needs now is:

  • An Azure AD account with read access to the Azure Bastion resource – no need for read to the hub or even the spoke with the VM!
  • The IP address for the machine they want to sign into – my design is limited to Azure VMs but on-premises static routes could be added to the Bastion hub route table.
  • A username and password with login rights to the virtual machine or computer.

Digital Transformation Is Not Just A Tech Thing

In this post, I want to discuss why many businesses fail to get what they expect from The Cloud – why their “digital transformation” (or “cloud transformation”) fails.

Cloud Concerns

We’ve all seen the surveys. CIOs are scared about lots of things when it comes to cloud migration/adoption. Costs might overrun. Security might be insufficient. Skills are in short supply. I’m afraid to say those are all concerns – manageable ones. The big question is rarely discussed until it is too late: what are we really getting into?

Failure Starts In The Middle

Many cloud adoption projects start at the wrong place in the organisation. Instead of an instruction coming with direction and authority from the C-suite (CEO, CTO, CSO, CIO, etc), IT management, typically in Operations, make the decision to go to The Cloud for mundane reasons such as “try to reduce costs” or “avoid doing another hardware upgrade”.

Operations go ahead and build (or request a consultant to deliver) what they know: a centrally managed, locked-down environment. You know what I mean; a rigid environment that complies with old ITIL-style processes from the early 2000s. If a developer needs something, log a ticket, and we’ll get around to it.

Meanwhile, developers hear that The Cloud is coming and they imagine a world where there is self-service and bottomless pits of compute and storage. Oh! And less waiting on tickets logged with Operations. And the C-suite gets a visit from AWS or Microsoft and is told about how agile and disruptive their businesses will become. Super!

The Age-Old Battle Continues

The Cloud landing zone is built and developers are given “access” (if we can call it that) to their new virtual workspace, only to find that they can create limited quantities and sizes of (breath) virtual machines. Modern platforms are nowhere to be found. The ability to create is locked away behind custom permissions roles. If they want something, to do their job for the business, they need to log a ticket and wait – just how is this any different from the old VMware platform they probably ran on before The Cloud?

And the business is digitally transformed. Well, no, not really. Some SAP stuff might be running on Azure hardware and VMs now. And some databases might have been moved to The Cloud. But that’s about it. None of the agility or disruption happens.

What Went Wrong?

The issue began right at the start. I’ll give you an example. A Microsoft account manager (or whatever they are called this financial year), a Microsoft partner, and an IT operations manager have a meeting. This sounds like a bad joke so far, and I promise that no one will laugh. The Microsoft account manager offers X dollars to perform an assessment. Someone will come in, scan the VMware machines, write a report that says “here is the TCO comparison” and “you’d be silly not to get started now”. So the Operations manager gives the go-ahead and the direction of what to do.

That contrasts greatly with the Microsoft Cloud Adoption Framework (CAF). Phase 1 (Strategy) of the CAF is all about getting business motivations from the C-suite that can be shaped into the direction in phase 2 (Plan) where the tech stuff begins – including the (digital estate) assessment. And the Plan phase is all about people and process. Ahh – people (skills) and process (change). This is where digital transformation begins. We haven’t even gotten to the part where things are deployed (phase 3, Ready) in The Cloud yet because we don’t know what “shape” the organisation will be until business motivations tell the IT departments what is expected from The Cloud.

The Microsoft Azure Cloud Adoption Framework

So What Is Digital Transformation?

In short, digital transformation should change:

  1. Process: Legacy methods of creating & running IT systems may not suite the business, especially if self-service, agility, and disruption and required. Look at the organisations that have shaken up different verticals in different industries and service sectors. Their IT departments are very different to the vertical pillars that you may recognise in your organisation. The order to change came from the top where the authority resides.
  2. People: How people are organised may need to change. The skills they possess must change. Roles must be identified and skills must be developed before the project, not during or even after some handover phase from a consulting company. A budget and time for training, possibly even a budget to recruit additional bodies requires authority that can only come from the top.
  3. Technology: It’s easier to change technology than to change process and people. The problem is what to change it to. The shape of the technology must reflect the people and process patterns, otherwise it is not fit for purpose.

Az Module Scripts in GitHub Actions

In this post, I will show how to run Azure Az module scripts as tasks in a GitHub action workflow. Working examples can be found in my GitHub AzireFirewall/DevSecOps repo which is the content for my DevSecOps articles.

Credit to my old colleague Alan Kinane for getting me started with his post, Azure CI/CD with ARM templates and GitHub Actions.

Why Use A Script?

One can do some simple deployment tasks, but a script offers some advantages:

  • A task that runs a deployment
  • A simple PowerShell/Azure CLI task that runs an inline script

But you might want something that does more. For example, you might want to do some error checking. Or maybe you are going to use a custom container and execute complex tasks from it. In my case, I wanted to do lots of error checking and give myself the ability to wrap scripts around my deployments.

My Example

For this post, I will use the hub deployment from my GitHub AzireFirewall/DevSecOps repo – this deploys a VNet-based (legacy) hub in an Azure hub & spoke architecture.. There are a number of things you are going to need. I’ve just rorganised and updated the main branch to support both Azure DevOps pipelines (/.pipelines) and GitHub actions (/.github).

Afterwards, I will explain how the action workflow calls the PowerShell script.

GitHub Repo

Set up a repository in GitHub. Copy the required files into the repo. In my example, there are four folders:

  • platform: This contains the files to deploy the hub in an Azure subscription. In my example, you will find bicep files with JSON parameter files.
  • scripts: This folder contains scripts used in the deployment. In my example, deploy.ps1 is a generic script that will deploy an ARM/Bicep template to a selected subscription/resource group.
  • .github/workflows: This is where you will find YAML files that create workflows in GitHub actions. Any valid file will automatically create a workflow when there is a sucessful merge. My example contains hub.yaml to execute the script, deploy.ps1.
  • .pipelines: This contains the files to deploy the code. In my example, you will find a YAML file for a DevOps pipeline called hub.yaml that will execute the script, deploy.ps1.

You can upload the files into the repo or sync using Git/VS Code.

Azure AD App Registration (Service Principal or SPN)

You will require an App Registration; this will be used by the GitHub workflow to gain authorised access to the Azure subscription.

Create an App Registation in Azure AD. Create a secret and store that secret (Azure Key Vault is a good location) because you will not be able to see the secret after creation. Grant the App Registration Owner rights to the Azure subscription (as in my example) or to the resource groups if you prefer that sort of deployment.

Repo Secret

One of the features that I like a lot in GitHub is that you can store secrets at different levels (organisation, repo, environment). In my example, I am storing the secrets for using the the Service Principal in the repo, making it available to the workflow(s) that are in this repo only.

Open the repo, browse to Settings, and then go to Secrets. Create a new Repository Secret called AZURE_CREDENTIALS with the following struture:

{
  "tenantId": "<GUID>",
  "subscriptionId": "<GUID>",
  "clientId": "<GUID>",
  "clientSecret": "<GUID>"
}

Create the Workflow

GitHub makes this task easy. You can into your Repo>Actions and create a workflow from a template. Or if you have a YAML file that defines your workflow, you can place it into your repo in /.github/workflows. When that change is merged, GitHub will automatically create a workflow for you.

Tip: To delete a workflow, rename or delete the associated file and merge the change.

Logging In

The workflow must be able to sign into Azure. There are plenty of examples out there but you need to accomplish 3 things:

  1. Log into Azure
  2. Enable the Az modules

The following code in the workflow accomplishes those three tasks:

      - name: Login via Az module
        uses: azure/login@v1
        with:
          creds: ${{secrets.AZURE_CREDENTIALS}}
          enable-AzPSSession: true

The line uses: azure/login@v1 enables and Azure sign-in. Note that the with statement selects the credentials that we previously added to the repo.

The line enable-AzPSSession: true enables the Az PowerShell modules. With that, we are signed in and have all we need to execute Azure PowerShell cmdlets.

Executing A PowerShell Script

A workflow has a section called jobs; in here, you can create multiple jobs that share something in common, such as a login. Each job is made up of steps. Steps perform actions such as checking out code from the repo, logging into Azure, and executing a deployment task (running a PowerShell script, for example).

I can create a PowerShell script that does lots of cool things and store it in my repo. That script can be edited and managed by change control (pull requests) just like my code that I’m deploying. There is an example of this below:

The PowerShell script step running in a GitHub action
      - name: Deploy Hub
        uses: azure/powershell@v1
        with:
          inlineScript: |
            .\scripts\deploy.ps1 -subscriptionId "${{env.hubSub}}" -resourceGroupName "${{env.hubNetworkResourceGroupName}}" -location "${{env.location}}" -deployment "hub" -templateFile './platform/hub.bicep' -templateParameterFile './platform/hub-parameters.json'
          azPSVersion: "latest"

The task is running “PowerShell v1” which will allow us to run an inline script using the sign-in that was previously created in the job (the login step). The configuration of this step is really simple – you specify the PowerShell cmdlets to run. In my example, it executes the PowerShell script and passes in the parameter values, some of which are stored as environment variables at the workflow level.

My script is pretty generic. I can have:

  • Multiple Bicep files/JSON parameter files
  • Multiple target scopes

I can create a PowerShell step for each deployment and use the parameters to specialise the execution of the script.

Running multiple PowerShell script tasks in a GitHub workflow

The PowerShell Script

The full code for the script can be found here. I’m going to focus on a few little things:

The Parameters

You can see in the above example that I passed in several parameters:

  • subscriptionId: The ID of the subscription to deploy the code to. This does not have to be the same as the default subscription specified in the Service Connction. The Service Principal used by the pipeline must have the required permissions in this subcsription.
  • resourceGroupName: The name of the resource group that the deployment will go into. My script will create the resource group if required.
  • location: The Azure region of the resource group.
  • deploymentName: The name of the ARM deployment that will be created in the resource group for the deployment (remember that Bicep deployments become ARM deployments).
  • templateFile: The path to the template file in the pipeline container.
  • templateParameterFile: The path to the parameter file for the template in the pipeline container.

Each of those parameters is identically named in param () at the start of the PowerShell script and those values specialise the execution of the generic script.

Outputs

You can use Write-Host to output a value from the script to appear in the console of the running job. If you add -ForegroundColor then you can make certain messages, such as errors or warnings, stand out.

Beware of Manual Inputs

Some PowerShell commands might want a manual input. This is not supported in a pipeline and will terminate the pipeline with an error. Test for this happening and use code logic wrapped around your cmdlets to prevent it from happening – this is why a file-based script is better than a simple/short inline script, even to handle a situation like creating a resource group.

Try/Catch

Error handling is a big deal in a hands-off script. You will find that 90% of my script is checking for things and dealing with unwanted scenarios that can happen. A simple example is a resource group.

An ARM deployment (remember this includes Bicep) must go into a resource group. You can just go ahead and write the one-liner to create a resource group. But what happens when you update the code, the script re-runs and sees the resource group is already there? In that scenario, a manual input will appear (and fail the pipeline) to confirm that you want to continue. So I have an elaborate test/to process:

if (!(Get-AzResourceGroup $resourceGroupName -ErrorAction SilentlyContinue))
{ 
  try 
  {
    # The resource group does not exist so create it
    Write-Host "Creating the $resourceGroupName resource group"
    New-AzResourceGroup -Name $resourceGroupName -Location $location -ErrorAction SilentlyContinue
  }
  catch 
  {
    # There was an error creating the resoruce group
    Write-Host "There was an error creaating the $resourceGroupName resource group" -ForegroundColor Red
    Break
  }
}
else
{
  # The resoruce group already exists so there is nothing to do
  Write-Host "The $resourceGroupName resource group already exists"
}

Conclusion

Once you know how to do it, executing a script in your pipeline is easy. Then your PowerShell knowledge can take over and your deployments can become more flexible and more powerful. My example executes ARM/Bicep deployments. Yours could do a PowerShell deployment, add scripted configurations to a template deployment, or even run another language like Terraform. The real thing to understand is that now you have a larger scripting toolset available to your automated deployments.

Az Module Scripts in DevOps Pipelines

In this post, I will show how to run Azure Az module scripts as tasks in an Azure DevOps pipeline. Working examples can be found in my GitHub AzireFirewall/DevSecOps repo which is the content for my DevSecOps articles.

Why Use A Script?

One can use simple deployment tasks in a DevOps pipeline:

  • A task that runs a deployment
  • A simple PowerShell/Azure CLI task that runs an inline script

But you might want something that does more. For example, you might want to do some error checking. Or maybe you are going to use a custom container (Azure Container Registry) and execute complex tasks from it. In my case, I wanted to do lots of error checking and give myself the ability to wrap scripts around my deployments.

The First Obstacle: Documentation

Azure DevOps documentation is notorious for being:

  • Bad
  • Out of date
  • Hard to find
  • Incomplete

The article you need to get started on using PowerShell can be found here. There is a Hello World example that shows how to pass in two parameters to a PowerShell script. I used that as the basis of my deployment – but it is not enough! I will fix that here.

The Second Obstacle: Examples

The DevOps world is very much a closed box. There’s lots of people doing stuff, but finding working examples is a nightmare. Once again, I will fix that here. The goal is to:

  1. Store your code in an Azure DevOps repo
  2. Create an Azure DevOps pipeline to deploy that code to Azure. It will authorised against the Azure subscription (or resource groups) using an App Registration that is stored in DevOps as a Service Connection.
  3. The pipeline will execute a PowerShell script to deploy code from the DevOps repo into your subscription (or resource groups).

My Example

For this post, I will use the hub deployment from my GitHub AzireFirewall/DevSecOps repo – this deploys a VNet-based (legacy) hub in an Azure hub & spoke architecture.. There are a number of things you are going to need.

Afterwards, I will explain how the pipeline calls the PowerShell script.

DevOps Repo

Set up a repository in Azure DevOps. Copy the required files into the repo. In my example, there are two folders:

  • platform: This contains the files to deploy the hub in an Azure subscription. In my example, you will find bicep files with JSON parameter files.
  • scripts: This folder contains scripts used in the deployment. In my example, deploy.ps1 is a generic script that will deploy an ARM/Bicep template to a selected subscription/resource group.
  • .pipelines: This contains the files to deploy the code. In my example, you will find a YAML file for a DevOps pipeline called hub.yaml that will execute the script, deploy.ps1.
  • .github/workflows: This is where you will find YAML files that create workflows in GitHub actions. Any valid file will automatically create a workflow when there is a sucessful merge. My example contains hub.yaml to execute the script, deploy.ps1.

You can upload the files into the repo or sync using Git/VS Code.

Azure AD App Registration (Service Principal or SPN)

You will require an App Registration; this will be used by the Azure DevOps pipeline to gain authorised access to the Azure subscription.

Create an App Registation in Azure AD. Create a secret and store that secret (Azure Key Vault is a good location) because you will not be able to see the secret after creation. Grant the App Registration Owner rights to the Azure subscription (as in my example) or to the resource groups if you prefer that sort of deployment.

Service Connection

In your Azure DevOps project, browse to Project Settings > Service Connections. Create a new Service Connection of the Azure Resource Manager type. Select Service Principal (Manual) and enter the required details:

  • Subscription ID: The ID of the subscription that will be the default subscription when the Service Principal signs in.
  • Subscription Name: The name of the subscription that will be the default subscription when the Service Principal signs in.
  • Service Principal ID: The Client ID of the App Registration (see it’s Overview page).
  • Service Principal Key: The secret of the App Registration that you should have saved.
  • Tenant ID: The ID of the Azure AD tenant. You can get this from the Overview of the App Registration.
  • Service Connection Name: The name of the Service Connection; a naming standard helps here. For example, I name it after the scope of the deployment (the target subscription name). Remember this name because it will be used by the YAML file (a value called
    azureSubscription) to create the pipeline. In my example, the service connection is called “hub”.

Hit Verify And Save – DevOps will verify that the Service Principal can sign in. You should double-check that it has rights over your desired scope in Azure (subscription in my example).

Create the Pipeline

In my example, the hard work is done. A YAML file defines the pipeline; you just need to import the YAML file to create the pipeline.

Go back to your DevOps project, browse to Pipelines, and click New Pipeline. Choose Azure Repos Git as the code location. Select your repo and choose Existing Azure Pipelines YAML File. Use the dropdown list box to select the YAML file – /.pipelines/devops-hub.yml in my case. Save the pipeline and it will run. If you go into the running job you should see a prompt, asking you to authorise the pipeline to use the “hub” service connection.

 

The pipeline will execute a task, that in turn, will run a PowerShell script. That PowerShell script takes in several parameters that tell the script what to deploy (bicep and parameter files), where to deploy it from (the temporary location where the files are downloaded into the pipeline container), and where to deploy it to (the subscription/resource group).

Executing A PowerShell Script

A pipeline has a section called steps; in here, you create a task for each job that you want to run. For example, I can execute an Azure CLI task, a PowerShell task that runs one/a few lines of inline code, or a PowerShell task that executes a PowerShell script from a file. It’s that last one that is interesting.

I can create a PowerShell script that does lots of cool things and store it in my repo. That script can be edited and managed by change control (pull requests) just like my code that I’m deploying. There is an example of this below:

The PowerShell script job running in Azure DevOps
steps:

# Deploy Hub

- task: AzurePowerShell@5
  inputs:
    azureSubscription: $(azureServiceConnection)
    ScriptType: 'FilePath'
    ScriptPath: $(System.DefaultWorkingDirectory)/.pipelines/deploy.ps1
    ScriptArguments: > # Use this to avoid newline characters in multiline string
      -subscriptionId $(hubSub)
      -resourceGroupName $(hubNetworkResourceGroupName)
      -location $(location)
      -deployment "hub"
      -templateFile $(System.DefaultWorkingDirectory)"/platform/hub.bicep"
      -templateParameterFile $(System.DefaultWorkingDirectory)"/platform/hub-parameters.json"
    azurePowerShellVersion: 'LatestVersion'
  displayName: 'Deploy Hub'

The tasks is running “PowerShell v5” (see AzurePowerShell@5). That’s an important thing to note. The Microsoft documentation for running PowerShell shows PowerShell v2, and that does not support the Az modules, which is pretty pointless! PowerShell v4 added the Az modules.

The azureSubscription value refers to the Service Connection that we created earlier, authorising the pipeline against the desired target scope.

ScriptType is set to FilePath (not inline) so I can run a PowerShell script from a file. That requires me to use ScriptPath to define where the script is.

When the pipeline runs, it is executed in a container (defined earlier in the YAML file as ubuntu-latest in my example, Linux to speed things up). The files in the repo are downloaded to a working folder. That location is saved as $(System.DefaultWorkingDirectory). I can then add the relative location of the PowerShell script from the repo ( /.pipelines/deploy.ps1 ) to that path so the pipeline can find the script in the container.

My script is pretty generic. I can have:

  • Multiple Bicep files/JSON parameter files
  • Multiple target scopes

I can create a PowerShell task for each deployment and use the parameters to specialise the execution of the script.

Running multiple PowerShell script tasks in a DevOps pipeline

We wrap up the task by specifying the PowerShell version to use and the display name for the task in the DevOps job console.

The PowerShell Script

The full code for the script can be found here. I’m going to focus on a few little things:

The Parameters

You can see in the above example that I passed in several parameters:

  • subscriptionId: The ID of the subscription to deploy the code to. This does not have to be the same as the default subscription specified in the Service Connction. The Service Principal used by the pipeline must have the required permissions in this subcsription.
  • resourceGroupName: The name of the resource group that the deployment will go into. My script will create the resource group if required.
  • location: The Azure region of the resource group.
  • deploymentName: The name of the ARM deployment that will be created in the resource group for the deployment (remember that Bicep deployments become ARM deployments).
  • templateFile: The path to the template file in the pipeline container.
  • templateParameterFile: The path to the parameter file for the template in the pipeline container.

Each of those parameters is identically named in param () at the start of the PowerShell script and those values specialise the execution of the generic script.

Outputs

You can use Write-Host to output a value from the script to appear in the console of the running job. If you add -ForegroundColor then you can make certain messages, such as errors or warnings, stand out.

Beware of Manual Inputs

Some PowerShell commands might want a manual input. This is not supported in a pipeline and will terminate the pipeline with an error. Test for this happening and use code logic wrapped around your cmdlets to prevent it from happening – this is why a file-based script is better than a simple/short inline script, even to handle a situation like creating a resource group.

Try/Catch

Error handling is a big deal in a hands-off script. You will find that 90% of my script is checking for things and dealing with unwanted scenarios that can happen. A simple example is a resource group.

An ARM deployment (remember this includes Bicep) must go into a resource group. You can just go ahead and write the one-liner to create a resource group. But what happens when you update the code, the script re-runs and sees the resource group is already there? In that scenario, a manual input will appear (and fail the pipeline) to confirm that you want to continue. So I have an elaborate test/to process:

if (!(Get-AzResourceGroup $resourceGroupName -ErrorAction SilentlyContinue))
{ 
  try 
  {
    # The resource group does not exist so create it
    Write-Host "Creating the $resourceGroupName resource group"
    New-AzResourceGroup -Name $resourceGroupName -Location $location -ErrorAction SilentlyContinue
  }
  catch 
  {
    # There was an error creating the resoruce group
    Write-Host "There was an error creaating the $resourceGroupName resource group" -ForegroundColor Red
    Break
  }
}
else
{
  # The resoruce group already exists so there is nothing to do
  Write-Host "The $resourceGroupName resource group already exists"
}

Conclusion

Once you know how to do it, executing a script in your pipeline is easy. Then your PowerShell knowledge can take over and your deployments can become more flexible and more powerful. My example executes ARM/Bicep deployments. Yours could do a PowerShell deployment, add scripted configurations to a template deployment, or even run another language like Terraform. The real thing to understand is that now you have a larger scripting toolset available to your automated deployments.