Azure / Azure/azure-linux-extensions

VMEncryption on CentOS is broken after recent update to the VMEncryption extension

Open
#564 4 comments 0 reactions 1 assignee Claimed by @vermashi View on GitHub
Dominant language
Python
Stars
333
Forks
278
Avg merge
2d 9h
Merged PRs (30d)
4

Description

This patch - 27757d457c93b94a54a16519c5e673ffcd797831 which was recently pushed out seems to have broken encrypted CentOS VM's.

If the VM is stopped/started or restarted, then very frequently it will fail to boot. Looking in the boot diagnostics, the boot gets stuck at this point:

> Please enter passphrase for disk Virtual_Disk (osencrypt)

Here is the boot log: [serialconsole.log](https://github.com/Azure/azure-linux-extensions/files/1922402/centos72.9fbf891b-0dc4-4b55-8a7e-13fe55ef52eb.serialconsole.log)

To reproduce:
- Provision a CentOS7.2
`"imageReference": {
"publisher": "OpenLogic",
"offer": "CentOS",
"sku": "7.2n",
"version": "latest"
}`
- Setup a KeyVault which is enabled for disk encryption
- Encrypt the VM like so:
`az vm encryption enable --resource-group --name --aad-client-id --aad-client-secret --disk-encryption-keyvault --key-encryption-key diskencryptionkey --volume-type OS`
- Wait until this command returns a status of VMRestartPending
`az vm encryption show --resource-group --name --query [osDisk] -o tsv`
- Restart the VM, and check the status (it might say to restart again)
- When the VMStatus is Encrypted
- Restart the VM or Stop/Start the VM a few times (Seems to happen more often with Stop/Start), at some point the VM will appear to be hung. Check the boot logs, and you will see that it is stuck on the message "Please enter passphrase for disk Virtual_Disk (osencrypt)"

This is a very serious problem for us (and probably anyone who is running CentOS with encrypted disks on Azure). We stop/start our non-prod environments on a schedule to save costs, this is no longer possible as the restarts are too unreliable. Production environments are now running with a huge risk that any planned or unplanned server restarts will not be recoverable.

There does not be any apparent way to work around this problem.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.