canonical / canonical/charm-integration-testing

test_deploy: pipelines-visualization rev 49 fails with ImagePullBackOff

Open
#799 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
6
Forks
1
Avg merge
1d 22h
Merged PRs (30d)
96

Description

**Note**: this is most likely a release/resource issue with the `pipelines-visualization` charm itself rather than a test-harness bug. Filing here first since it was surfaced via `test_deploy` triage; if confirmed, please transfer/re-file against the likely upstream repo (charmed-kubeflow `pipelines-visualization-operator`, verify exact repo name).

## Summary

`pipelines-visualization` revision **49** fails to deploy in every observed `test_deploy` run: the `target` unit's workload container enters `ImagePullBackOff` because the referenced OCI image (`registry.jujucharms.com/kubeflow-charmers/pipeli...`, truncated in Test Observer metadata) cannot be pulled. This is **5 of 5** untriaged `error`-only failures, plus **3 of 5** `blocked+error+waiting` combo failures — all the same artefact/revision. This is not charm logic; it is a broken/missing release resource.

## Environment

- Charm: `pipelines-visualization` revision **49**
- Test harness: `charm-integration-testing` `test_deploy` plan

## Evidence (reproducible CI runs — all same signature)

| test_result_id | test_execution_id | CI run |
|---|---|---|
| 11911650 | 686956 | https://github.com/canonical/charm-integration-testing/actions/runs/28281098566 |
| 11911524 | 686954 | https://github.com/canonical/charm-integration-testing/actions/runs/28280440977 |
| 11910943 | 686946 | https://github.com/canonical/charm-integration-testing/actions/runs/28277708276 |
| 11910817 | 686944 | https://github.com/canonical/charm-integration-testing/actions/runs/28276736227 |
| 11911758 | 686958 | https://github.com/canonical/charm-integration-testing/actions/runs/28281754429 (combo w/ ceph-mon/k8s) |
| 11911324 | 686952 | https://github.com/canonical/charm-integration-testing/actions/runs/28279566865 (combo w/ ceph-mon/k8s) |

Representative status line:
```
[error] target: unknown container reason "ImagePullBackOff": Back-off pulling image "registry.jujucharms.com/kubeflow-charmers/pipeli...
```

## Root cause hypothesis

The OCI resource attached to `pipelines-visualization` revision 49 either references an image tag that was deleted/never published to `registry.jujucharms.com`, or was pinned incorrectly at release time (typo, wrong digest, or an internal/unpublished tag).

## Reproduction steps

1. Deploy `pipelines-visualization` revision 49 via its `test_deploy` bundle.
2. Observe `target/0` enter `error` with container reason `ImagePullBackOff`.
3. Inspect the exact image reference: `kubectl describe pod ` in the test's k8s model, or check `charmcraft resources pipelines-visualization --channel=` for the OCI resource attached to revision 49.
4. Attempt to pull the referenced image directly (`docker pull ` / `oras manifest fetch `) to confirm it's unavailable.

## Suggested fix

Re-push a valid OCI resource for revision 49 (or cut a new revision with a corrected image reference) and re-release. This is a release-process fix, not a code change.

## Suggested regression test

Add a pre-release CI check that resolves every OCI resource reference before a new revision is published, so a broken image reference is caught before it reaches `test_deploy` rather than after.

Suggested labels: `bug`, `release-blocker`, `test_deploy`, `pipelines-visualization`

Contributor guide

Open the contributing guide

Research direction

Start with the `test_deploy` plan and the linked CI runs, then inspect the failing pod with `kubectl describe pod` and resolve the OCI resource using `charmcraft resources`. Attempt the referenced image with `docker pull` or `oras manifest fetch`. Done means confirming the unavailable resource and transferring or re-filing the issue against the correct upstream charm repository for a corrected release.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, kubernetes, python
Domain
ci-cd, devops, release
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.