canonical / canonical/charm-integration-testing

test_downgrade_charm fails for self-signed-certificates rev 588 due to base change between revisions

Open
#466 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
6
Forks
1
Avg merge
1d 22h
Merged PRs (30d)
96

Description

**Note: This issue was generated with AI assistance (GitHub Copilot) based on automated log analysis and triage.**
Filed by @canonical/solutions-qa

## Summary

`test_downgrade_charm` consistently fails for `self-signed-certificates` rev 588 across two tracks with two distinct failure modes, both caused by the charm changing its supported base between revisions.

## Failure Mode 1: Base incompatibility during downgrade (artefact 121975, track 1/stable)

The test locates rev 586 as the historical downgrade target (from execution 134136), but the `juju refresh` command fails:

```
jubilant._juju.CLIError: Command '['juju', 'refresh', ..., 'target', '--channel', '1/stable', '--revision', '586']' returned non-zero exit status 1.
Stderr:
ERROR cannot upgrade from single base "ubuntu@22.04" charm to a charm supporting ["ubuntu@24.04"]. Use --force-series to override.
```

Rev 588 was deployed with `base: ubuntu@22.04` in the generated bundle, but its charm metadata advertises `ubuntu@24.04` support. Rev 586 only supports `ubuntu@22.04`. Juju refuses the cross-base refresh.

This failure is reproducible and appeared in executions 136827 (test result 24879) and 136825 (test result 24847).

Test Observer UI: https://test-observer.canonical.com/#/charms/121975?testExecutionId=136827&testResultId=24879

## Failure Mode 2: No historical revision found (artefact 122025, track 14)

```
Failed: Unable to find a historical revision with a passing test_deploy result for charm 'self-signed-certificates' in channel '1/stable'.
```

No prior revision with a passing `test_deploy` result exists in Test Observer for this charm+channel combination (likely because the base incompatibility issue affects all historical revisions once rev 588 was released).

Test Observer UI: https://test-observer.canonical.com/#/charms/122025?testExecutionId=136807&testResultId=24668

## Evidence

| Artefact ID | Track | Execution ID | Test Result ID |
|-------------|-------|-------------|----------------|
| 121975 | 1/stable | 136827 | 24879 |
| 121975 | 1/stable | 136825 | 24847 |
| 122025 | 14 | 136807 | 24668 |

CI run: https://github.com/canonical/charm-integration-testing/actions/runs/24459461460

## Root Cause

`self-signed-certificates` rev 588 changed its supported base (adds or switches to `ubuntu@24.04`), creating a base mismatch when `test_downgrade_charm` attempts to `juju refresh` back to rev 586 which only supports `ubuntu@22.04`. Juju blocks the operation without `--force-series`.

The `test_downgrade_charm` test framework does not handle this scenario.

## Suggested Fix

1. Before attempting `juju refresh`, check if the bases of the current revision and the historical target revision are compatible.
2. If they are incompatible, skip the test with an informative message rather than failing.
3. In the "no viable historical revision" case, also skip rather than fail (to avoid cascading test state corruption that SKIPs `test_deploy_target_old_revision` and `[injected] test_upgrade_charm`).

Contributor guide

Open the contributing guide

Research direction

Start at the test_downgrade_charm entry point and trace the historical revision lookup and the juju refresh to the target revision. Reproduce the failures from CI run 24459461460, then verify that incompatible bases and missing passing historical revisions are skipped with informative messages rather than failing or cascading into later tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.