canonical / canonical/charm-integration-testing
test_downgrade_charm fails for self-signed-certificates rev 588 due to base change between revisions
- Dominant language
- Python
- Stars
- 6
- Forks
- 1
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 96
Description
**Note: This issue was generated with AI assistance (GitHub Copilot) based on automated log analysis and triage.**
Filed by @canonical/solutions-qa
## Summary
`test_downgrade_charm` consistently fails for `self-signed-certificates` rev 588 across two tracks with two distinct failure modes, both caused by the charm changing its supported base between revisions.
## Failure Mode 1: Base incompatibility during downgrade (artefact 121975, track 1/stable)
The test locates rev 586 as the historical downgrade target (from execution 134136), but the `juju refresh` command fails:
```
jubilant._juju.CLIError: Command '['juju', 'refresh', ..., 'target', '--channel', '1/stable', '--revision', '586']' returned non-zero exit status 1.
Stderr:
ERROR cannot upgrade from single base "ubuntu@22.04" charm to a charm supporting ["ubuntu@24.04"]. Use --force-series to override.
```
Rev 588 was deployed with `base: ubuntu@22.04` in the generated bundle, but its charm metadata advertises `ubuntu@24.04` support. Rev 586 only supports `ubuntu@22.04`. Juju refuses the cross-base refresh.
This failure is reproducible and appeared in executions 136827 (test result 24879) and 136825 (test result 24847).
Test Observer UI: https://test-observer.canonical.com/#/charms/121975?testExecutionId=136827&testResultId=24879
## Failure Mode 2: No historical revision found (artefact 122025, track 14)
```
Failed: Unable to find a historical revision with a passing test_deploy result for charm 'self-signed-certificates' in channel '1/stable'.
```
No prior revision with a passing `test_deploy` result exists in Test Observer for this charm+channel combination (likely because the base incompatibility issue affects all historical revisions once rev 588 was released).
Test Observer UI: https://test-observer.canonical.com/#/charms/122025?testExecutionId=136807&testResultId=24668
## Evidence
| Artefact ID | Track | Execution ID | Test Result ID |
|-------------|-------|-------------|----------------|
| 121975 | 1/stable | 136827 | 24879 |
| 121975 | 1/stable | 136825 | 24847 |
| 122025 | 14 | 136807 | 24668 |
CI run: https://github.com/canonical/charm-integration-testing/actions/runs/24459461460
## Root Cause
`self-signed-certificates` rev 588 changed its supported base (adds or switches to `ubuntu@24.04`), creating a base mismatch when `test_downgrade_charm` attempts to `juju refresh` back to rev 586 which only supports `ubuntu@22.04`. Juju blocks the operation without `--force-series`.
The `test_downgrade_charm` test framework does not handle this scenario.
## Suggested Fix
1. Before attempting `juju refresh`, check if the bases of the current revision and the historical target revision are compatible.
2. If they are incompatible, skip the test with an informative message rather than failing.
3. In the "no viable historical revision" case, also skip rather than fail (to avoid cascading test state corruption that SKIPs `test_deploy_target_old_revision` and `[injected] test_upgrade_charm`).
Contributor guide
Research direction
Start at the test_downgrade_charm entry point and trace the historical revision lookup and the juju refresh to the target revision. Reproduce the failures from CI run 24459461460, then verify that incompatible bases and missing passing historical revisions are skipped with informative messages rather than failing or cascading into later tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100