canonical / canonical/haproxy-operator
[Bug]: Lack of logrotate configuration causes disk exhaustion and unit crashes for high-traffic ingresses
- Dominant language
- Python
- Stars
- 4
- Forks
- 12
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 22
Description
### Bug Description
We are experiencing disk exhaustion issues on units running the haproxy charm when handling high-traffic ingresses. Because the charm currently does not expose any configuration for logrotate, it falls back to the default log rotation policy, which only rotates logs once a week.
For high-volume environments, this weekly rotation is far too slow, causing the HAProxy logs to fill up the entire disk before the rotation is ever triggered. When the disk is filled, the charm goes into error and stops serving.
We already expanded the disk size but this is not a sustainable solution, we have and application that stops every 2 days. We need the ability to configure log rotation (e.g., size-based rotation, daily/hourly frequency, or capping retention) directly through the charm.
### Impact
Medium (functionality degraded, workaround exists)
### Impact Rationale
_No response_
### To Reproduce
1. Deploy the haproxy charm and relate it to workloads as an ingress.
2. Direct a high and continuous volume of traffic to the ingress.
3. Monitor the disk usage on the HAProxy unit.
4. Observe that the HAProxy log files grow unbounded over the course of the week because log rotation only runs weekly.
5. Once the disk fills up, observe the unit going into error.
### Environment
channel=2.8/edge
revision=537
running on canonical-k8s
### Relevant log output
```shell
2026-08-14 05:44:13 INFO juju.worker.uniter.operation runhook.go:186 ran "haproxy-route-relation-created" hook (via hook dispatching script: dispatch)
2026-08-14 05:44:14 ERROR unit.ingress-ps7-certification/15.juju-log server.go:405 haproxy-route:1220: 2 validation errors for RequirerApplicationData
service
Field required [type=missing, input_value={}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.13/v/missing
ports
Field required [type=missing, input_value={}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.13/v/missing
Traceback (most recent call last):
File "/var/lib/juju/agents/unit-ingress-ps7-certification-15/charm/lib/charms/haproxy/v2/haproxy_route.py", line 272, in load
return cls.model_validate_json(json.dumps(data))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/juju/agents/unit-ingress-ps7-certification-15/charm/venv/lib/python3.12/site-packages/pydantic/main.py", line 782, in model_validate_json
return cls.__pydantic_validator__.validate_json(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 2 validation errors for RequirerApplicationData
service
Field required [type=missing, input_value={}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.13/v/missing
ports
Field required [type=missing, input_value={}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.13/v/missing
2026-08-14 05:44:14 ERROR unit.ingress-ps7-certification/15.juju-log server.go:405 haproxy-route:1220: Invalid requirer application data for remote-dcefa3f1833d4dc18e0cd06dcb274f63
2026-08-14 05:44:14 ERROR unit.ingress-ps7-certification/15.juju-log server.go:405 haproxy-route:1220: 2 validation errors for RequirerApplicationData
service
Field required [type=missing, input_value={}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.13/v/missing
ports
Field required [type=missing, input_value={}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.13/v/missing
Traceback (most recent call last):
File "/var/lib/juju/agents/unit-ingress-ps7-certification-15/charm/lib/charms/haproxy/v2/haproxy_route.py", line 272, in load
return cls.model_validate_json(json.dumps(data))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/juju/agents/unit-ingress-ps7-certification-15/charm/venv/lib/python3.12/site-packages/pydantic/main.py", line 782, in model_validate_json
return cls.__pydantic_validator__.validate_json(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 2 validation errors for RequirerApplicationData
service
Field required [type=missing, input_value={}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.13/v/missing
ports
Field required [type=missing, input_value={}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.13/v/missing
2026-08-14 05:44:14 ERROR unit.ingress-ps7-certification/15.juju-log server.go:405 haproxy-route:1220: Invalid requirer application data for remote-dcefa3f1833d4dc18e0cd06dcb274f63
2026-08-14 05:44:16 INFO unit.ingress-ps7-certification/15.juju-log server.go:405 haproxy-route:1220: ha integration is not ready, skipping.
```
### Additional context
_No response_
Contributor guide
Research direction
Start by locating the charm entry point that configures HAProxy logging and the current logrotate setup. Review the issue's reproduction steps and determine how size, frequency, and retention should be exposed as charm configuration. Done means high-volume logs rotate before exhausting disk space, the unit remains healthy, and the behavior is covered by the repository's relevant tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100