canonical / canonical/observability-libs
k8s resource patch is losing the fight with juju
- Dominant language
- Python
- Stars
- 4
- Forks
- 8
- Avg merge
- 1h 26m
- Merged PRs (30d)
- 1
Description
### Bug Description
In some charms, we apply resource constraints on the charm container because we wanted fine-grained control over how much memory/cpu the charm container got without having to apply the same to the workload container(s).
juju 3.6.9 started setting a default memory limit (64M) **on the charm container**.
This breaks the service patch because we now get conflicts when we try to override it.
Our goal is to have per-container resource requests.
AFFECTED CHARMS:
🐛 loki-coordinator-k8s
🐛 mimir-coordinator-k8s
🐛 pyroscope-operators
🐛 tempo-operators
### To Reproduce
deploy anything using the service patch on juju 3.6.9
### Environment
juju 3.6.9
### Relevant log output
```shell
iunit-tempo-0: 10:15:28 ERROR unit.tempo/0.juju-log resource patcher failed with message 'Kubernetes resources patch failed: Apply failed with 1 conflict: conflict with "juju" using apps/v1: .spec.template.spec.contain
ers[name="charm"].resources.requests.memory'
unit-tempo-0: 10:15:28 INFO unit.tempo/0.juju-log HTTP Request: GET https://10.152.183.1/apis/apps/v1/namespaces/test-tempo/statefulsets/tempo "HTTP/1.1 200 OK"
unit-tempo-0: 10:15:28 INFO unit.tempo/0.juju-log HTTP Request: PATCH https://10.152.183.1/apis/apps/v1/namespaces/test-tempo/statefulsets/tempo?fieldManager=ResourcePatcher&dryRun=All "HTTP/1.1 409 Conflict"
unit-tempo-0: 10:15:29 ERROR unit.tempo/0.juju-log resource patcher failed with message 'Kubernetes resources patch failed: Apply failed with 1 conflict: conflict with "juju" using apps/v1: .spec.template.spec.contain
ers[name="charm"].resources.requests.memory'
```
### Additional context
Even if juju were to fix this, 3.6.9 would remain broken (and it'd take time). So we may need a release on our side with force=True so the charms aren't broken for now.
Our only stable track is 1/
We need to guarantee successful in-track upgrades.
https://github.com/juju/juju/issues/20446
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the service-patch failure on Juju 3.6.9 and inspect the resource patcher entry point; the issue does not name specific files or tests. Trace the conflict on the charm container's memory request and verify that the fix preserves per-container requests and successful in-track upgrades for the affected charms.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, python
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100