canonical / canonical/charm-integration-testing
[Investigation Needed] Is `ckf-1.10/stable` channel for Minio obsolete?
- Dominant language
- Python
- Stars
- 6
- Forks
- 1
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 96
Description
**Note: This issue was generated with AI assistance (GitHub Copilot) based on automated log analysis and triage.**
Filed by @canonical/solutions-qa
---
## `test_model_controller_migration` fails: minio rev 583 stuck in `waiting for container` after migration
### Summary
When running `test_model_controller_migration` with a bundle that includes `minio` (ckf-1.10/stable, rev 583), the test fails because minio enters an infinite loop of setting its pod spec then setting `waiting for container` status. The Kubernetes pod is **Running and Ready**, but Juju's workload status never transitions to `active`.
`minio` rev 583 (ckf-1.10/stable) uses the old **pod-spec model**. Does this make it obsolete?
### Environment
- **Target charm**: `opentelemetry-collector-k8s` 1/stable rev 31
- **Failing charm**: `minio` ckf-1.10/stable rev 583 (neighbour in bundle)
- **Juju version**: 3.6.21
- **Substrate**: Kubernetes (k8s-production)
- **Bundle**: `grafana-k8s` r180, `minio` r583, `argo-controller` r856, `opentelemetry-collector-k8s` r31
### Observed behaviour
After the model is migrated to a temporary controller and back, `minio/0` gets stuck in `waiting for container`:
App Version Status Scale Charm Channel Rev
minio res:oci-image@7f2474f waiting 1 minio ckf-1.10/stable 583
waiting for container
Unit Workload Agent Message
minio/0* waiting idle waiting for container
The Kubernetes pod (`describe-minio-0.txt`) shows all conditions `True` and status `Running`.
### Status log (from juju logs)
The charm's `update_status` hook runs every ~2 minutes and repeats this cycle without ever reaching `active`:
26 Apr 2026 01:45:22Z workload maintenance Setting pod spec
26 Apr 2026 01:45:22Z workload waiting waiting for container
26 Apr 2026 01:47:00Z workload maintenance Setting pod spec
26 Apr 2026 01:47:00Z workload waiting waiting for container
26 Apr 2026 01:49:09Z workload maintenance Setting pod spec
26 Apr 2026 01:49:09Z workload waiting waiting for container
...
### Root cause hypothesis
`minio` rev 583 (ckf-1.10/stable) uses the old **pod-spec model**. Its `update_status` hook unconditionally:
1. Calls `pod-spec-set` (even when the spec is unchanged)
2. Sets `waiting("waiting for container")`
After migration, the Juju controller sees the pod spec as unchanged, so it never fires the "container ready" event that would let the charm advance to `active`. The charm loops indefinitely.
The newer minio (on `main`) has been rewritten to use the Pebble/sidecar model and would not have this issue. It is not clear whether a fix needs to be backported to ckf-1.10/stable, or whether the test bundle should be updated to use a newer minio revision/channel.
### Steps to reproduce
1. Deploy the bundle: `grafana-k8s:2/stable`, `minio:ckf-1.10/stable`, `argo-controller:3.4/stable`, `opentelemetry-collector-k8s:1/stable` with the relations as described in the bundle
2. Run `test_model_controller_migration`
3. After migration completes, observe `minio/0` stuck in `waiting for container` despite the Kubernetes pod being Running and Ready
### Test Observer link
https://test-observer.canonical.com/#/charms/406485?testExecutionId=494558&testResultId=10699245
### Pattern
This failure also appears in other bundles that include `minio` ckf-1.10/stable:
- `argo-controller` executions 539208, 539236 (May 2026)
- `minio` execution 543279 (June 2026)
### Open questions
- Should ckf-1.10/stable minio be updated so `update_status` does not unconditionally call `pod-spec-set`?
- Or should the test bundle be updated to use a newer minio channel that uses the Pebble model?
- Is this a known incompatibility between pod-spec charms and `test_model_controller_migration`?
Contributor guide
Research direction
Start with the test_model_controller_migration entry point and reproduce it using the listed bundle and minio ckf-1.10/stable rev 583. Compare the Juju status logs with describe-minio-0.txt and determine whether the correction belongs in the test bundle or the minio channel. Done means the migration test no longer leaves minio/0 waiting despite a Ready pod.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, python
- Domain
- devops, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 38/100