In this post, I’ll show how I solved a failure, that occurred during an Azure Image Builder (Packer) build with a Ubuntu 20.04 image, which resulted in a bunch of errors that contained E: Could not open file /var/lib/apt/lists/ with a bunch of different file names.
Disclaimer
I am Linux-disabled. I started my career programming on UNIX but switched to being a Microsoft infrastructure person a year later – and that was a long time ago. I am not a frequent Linux user but I do acknowledge its existence and usefulness. In other words, I figured out a fix for me, but it might not be a fix for you.
The Problem
I was using Azure Image Builder, which is based on Packer, to allow the regular creation of a Ubuntu 20.04 image with the latest updates and bits for acting as the foundation of a self-hosted DevOps agent VM Scale Set in a secure Azure network.
I had simple needs:
- Install Unzip
- Install Terraform
What makes it different is that I need the installations to be non-interactive. Windows has a great community with that kind of challenge. After a lot of searching, I realise that Linux does not.
I set up the tasks in the image template and for a month, everything was fine. Images built and rebuilt. A few days ago, a weird issue started where the first version of a template build was fine, but subsequent builds failed. When I looked at the build log, I saw a series of errors when apt (the package installed) ran that started with:
E: Could not open file /var/lib/apt/lists/…
The Solution
I tried a lot of things, including:
apt-get update
apt-get upgrade -y
But guess, what – the errors just moved.
I was at the end of my tether when I decided to try something else. The apt package installation for WinZip worked some of the time. What was wrong the rest of the time? Time – that was the key word.
Something needed more time before I ran any apt commands. I decided to embed a bunch of sleep commands to let things in Ubuntu catchup with my build process.
I have two tasks that run before I install Terraform. The first prepares Linux:
{
"type": "Shell",
"name": "Prepare APT",
"inline": [
"echo ABCDEFG",
"echo sleep for 90 seconds",
"sleep 1m 30s",
"echo apt-get update",
"apt-get update",
"echo apt-get upgrade",
"apt-get upgrade -y",
"echo sleep for 90 seconds",
"sleep 1m 30s"
]
},
The second task installs WinZip and some other tools that assist with downloading the latest Terraform zip file:
{
"type": "Shell",
"name": "InstallPrereqs",
"inline": [
"echo ABCDEFG",
"echo sleep for 90 seconds",
"sleep 1m 30s",
"echo installing unzip",
"sudo apt install --yes unzip",
"echo installing jq",
"sudo snap install jq"
]
},
I’ve ran this code countless times yesterday and it worked perfectly. Sure, the sleeps slow things down, but this is a batch task that (outside of testing) I won’t be waiting on so I am not worried.