canonical / canonical/charm-integration-testing
prodstack6 fails to provision ubuntu@20.04 and ubuntu@22.04 machines — `ubuntu:22.04 openstack` and `ubuntu:20.04 openstack` environments completely broken
- Dominant language
- Python
- Stars
- 6
- Forks
- 1
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 96
Description
**Note: This issue was generated with AI assistance (GitHub Copilot) based on automated log analysis and triage.**
Filed by @canonical/solutions-qa
---
### Summary
The `juju:3/stable ubuntu:22.04 openstack` and `juju:3/stable ubuntu:20.04 openstack` environments on prodstack6 are unable to provision machines with ubuntu@20.04 or ubuntu@22.04 bases. Every `test_deploy` run since at least 2026-06-22 09:50 UTC has failed with the same Keystone image metadata error. The `juju:3/stable ubuntu:24.04 openstack` environment is unaffected. ubuntu@24.04 machines also provision successfully within the failing runs themselves.
### Observed Error
From Juju status at time of failure (consistent across all affected runs):
```
Machine State Address Inst id Base AZ Message
X down pending ubuntu@22.04 cannot get available image metadata: failed getting
published images metadata from keystone catalog:
cannot read inde...
```
The same error occurs for ubuntu@20.04. ubuntu@24.04 machines in the same deployments and availability zones provision without issue.
### Affected Environments
| Environment | Failures since 2026-06-22 | ubuntu@24.04 in same runs |
|---|---|---|
| `juju:3/stable ubuntu:22.04 openstack` | 27 | ✅ provisions fine |
| `juju:3/stable ubuntu:20.04 openstack` | 4 | ✅ provisions fine |
| `juju:3/stable ubuntu:24.04 openstack` | 0 | n/a |
### Scope
The issue is not specific to any charm. Affected test plans in `ubuntu:22.04 openstack` span `kubernetes-worker`, `kubernetes-control-plane`, `calico`, `canal`, `flannel`, `containerd`, `etcd`, `ceph-mon`, `ceph-osd`, `azure-integrator`, `gcp-integrator`, `easyrsa`, `manual-tls-certificates`, `lego`, and others. The `ubuntu:20.04 openstack` environment shows the same failure pattern.
Sample of affected executions (`ubuntu:22.04 openstack`):
| Execution | Test Plan | Failed bases |
|-----------|-----------|-------------|
| 332860 | kubernetes-worker / ceph-dashboard | ubuntu@22.04 ×3 |
| 332858 | kubernetes-worker / apache2 | ubuntu@20.04, ubuntu@22.04 |
| 332854 | kubernetes-worker / apache2 (multi-bundle) | ubuntu@20.04, ubuntu@22.04 ×5 |
| 332852 | kubernetes-worker / kata | ubuntu@22.04 ×2 |
| 332832 | kubernetes-worker / calico | ubuntu@22.04 |
| 332824 | gcp-integrator / kubernetes-worker | ubuntu@20.04, ubuntu@22.04 ×2 |
| 332818 | azure-integrator / kubernetes-worker | ubuntu@22.04 ×2 |
| 332808 | lego / kubernetes-worker | ubuntu@22.04 ×4 |
Sample of affected executions (`ubuntu:20.04 openstack`):
| Execution | Test Plan | Failed bases |
|-----------|-----------|-------------|
| 332811 | self-signed-certificates / kubernetes-worker | ubuntu@20.04 ×2 |
| 332809 | manual-tls-certificates / kubernetes-worker | ubuntu@20.04 ×2, ubuntu@22.04 |
| 332807 | lego / kubernetes-worker | ubuntu@20.04 ×2, ubuntu@22.04 ×2 |
### Environment
- **Cloud/Region**: `target-cloud/prodstack6`
- **Juju version**: 3.6.24
- **Confirmed affected bases**: ubuntu@20.04, ubuntu@22.04
- **Unaffected**: ubuntu@24.04 (both as a dedicated environment and within the failing runs)
### Steps to Reproduce
Run any charm integration test that deploys an application with `base: ubuntu@22.04` or `base: ubuntu@20.04` against the `ubuntu:22.04 openstack` or `ubuntu:20.04 openstack` environments on prodstack6.
### Test Observer
https://test-observer.canonical.com/#/charms/333181?testExecutionId=332828&testResultId=11866320
Contributor guide
Research direction
Start by running any listed charm integration test against the affected prodstack6 environments and compare ubuntu@20.04 and ubuntu@22.04 provisioning with ubuntu@24.04. Investigate the repeated Keystone image metadata error using the affected execution details and Test Observer link; done means machines provision successfully for both affected bases without regressing ubuntu@24.04.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cloud, infrastructure, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100