canonical / canonical/charm-integration-testing
mimir-worker-k8s deployed with no role config: coordinator sees "Incoherent deployment", test_deploy times out
- Dominant language
- Python
- Stars
- 6
- Forks
- 1
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 98
Description
**Note: This issue was generated with AI assistance (GitHub Copilot) based on automated log analysis and triage.**
Filed by @canonical/solutions-qa
## Summary
`mimir-worker-k8s` is deployed by the test harness with no config options set. The charm requires at least one `role-*` config key to be set to `true` (e.g. `role-all=true`); without one it immediately self-blocks and never publishes valid data on the `mimir-cluster` relation. Its coordinator (`mimir-coordinator-k8s`) then detects an "incoherent deployment" and shuts itself down defensively. Both applications end up stuck `waiting: installing agent` until the test's scale-wait times out.
## Evidence
Test Observer result: https://test-observer.canonical.com/#/charms/409...?testExecutionId=704113&testResultId=13403432 (test_execution 704113, test_plan `integration/mimir-coordinator-k8s:self-metrics-endpoint/prometheus_scrape/prometheus-k8s:metrics-endpoint`)
`bundle.yaml` for the target model shows `mimir-worker-k8s` deployed with no `options:` block at all:
```yaml
mimir-worker-k8s:
charm: mimir-worker-k8s
channel: 2.17/stable
revision: 102
...
storage:
data: kubernetes,1,1024M
recovery-data: kubernetes,1,1024M
trust: true
```
`mimir-worker-k8s/0` status log:
```
31 Aug 2026 12:18:08Z workload blocked Missing relation to a coordinator charm
...
31 Aug 2026 12:50:59Z workload blocked Node offline: no role assigned. Please configure this worker to enable a role. For example: 'juju config mimir-worker-k8s role-all=true'.
```
`target` (mimir-coordinator-k8s) unit log, repeated on every `mimir-cluster-relation-joined`/`-changed` event:
```
ERROR juju-log mimir-cluster:1: failed to validate databag: {'egress-subnets': '...', 'ingress-address': '...', 'private-address': '...'}
Traceback (most recent call last):
...
pydantic_core._pydantic_core.ValidationError: 2 validation errors for ClusterRequirerUnitData
juju_topology
Field required [type=missing, input_value={}, input_type=dict]
address
Field required [type=missing, input_value={}, input_type=dict]
ERROR juju-log mimir-cluster:1: Incoherent deployment. target/0 will be shutting down. This likely means you are lacking some required roles in your workers. This charm will be unresponsive and refuse to handle any event until the situation is resolved by the cloud admin, to avoid data loss.
```
Test failure: `juju.backend.JujuWaitTimeoutError: Timed out while waiting for models: [...] to become idle (applications: ['mimir-worker-k8s', 'target'], units: ['mimir-worker-k8s/0', 'target/0'])`
## Root cause
`static/charm-overrides/` has no override file for `mimir-worker-k8s` (nor, as far as we could find, for sibling COS "coordinator/worker" charms like `loki-worker-k8s` or `tempo-worker-k8s`) declaring a required role config. Other charms in this repo already use the established pattern for exactly this kind of "must pick one of several mutually exclusive config values" requirement, e.g. `static/charm-overrides/mongodb-k8s.yaml`:
```yaml
configs:
role: [replication, config-server, shard, null]
constraints:
- 'bool(endpoint[config-server]) == (config[role] == "config-server")'
- 'bool(endpoint[sharding]) == (config[role] == "shard")'
```
Without an equivalent override for `mimir-worker-k8s` forcing e.g. `role-all: true` (or a per-role split matching how the bundle actually wires workers to the coordinator), the bundle-builder has no reason to ever set a role config, and the worker charm blocks by design.
This is a distinct failure mode from #837 (tempo-worker-k8s, which is an S3/minio access-denied issue) and from #919 (a bundle_builder_x bug in optional-config extraction, already fixed — not applicable here since no override/constraint exists for this charm at all).
## Suggested fix
Add `static/charm-overrides/mimir-worker-k8s.yaml` with a `configs` entry setting a role config (most simply `role-all: [true]` with a constraint forcing it to be set), following the pattern already used for `mongodb-k8s.yaml`. Worth auditing whether `loki-worker-k8s`/`tempo-worker-k8s` (if/when exercised as coordinator/worker pairs) need the same treatment.
Contributor guide
Research direction
Start with static/charm-overrides/mongodb-k8s.yaml and inspect how bundle wiring configures mimir-worker-k8s in the cited integration test. Add static/charm-overrides/mimir-worker-k8s.yaml using the established role-configuration pattern, checking whether sibling worker overrides are relevant. Done means the worker receives a role, publishes valid mimir-cluster data, and the test no longer times out.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, python
- Domain
- devops, testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 72/100