Elastic Agent on K8s - Improve logging on falling behind reading the event queue of K8s
- Dominant language
- Go
- Stars
- 12.7k
- Forks
- 5k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 364
Description
**Describe the enhancement:**
When low on resources, Elastic Agent used to implement K8s observability use cases might log:
```
{"log.level":"error","@timestamp":"2025-09-15T00:31:18.803Z","message":"I0915 00:31:16.372977 43 trace.go:236] Trace[997072019]: \"DeltaFIFO Pop Process\" ID:kube-system/konnectivity-agent-b5787dd87-h4z4k,Depth:12,Reason:slow event handlers blocking the queue (15-Sep-2025 00:31:11.464) (total time: 236ms):","component":{"binary":"metricbeat","dataset":"elastic_agent.metricbeat","id":"beat/metrics-monitoring","type":"beat/metrics"},"log":{"source":"beat/metrics-monitoring"},"ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2025-09-15T00:31:18.810Z","message":"Trace[997072019]: [236.621333ms] [236.621333ms] END","component":{"binary":"metricbeat","dataset":"elastic_agent.metricbeat","id":"beat/metrics-monitoring","type":"beat/metrics"},"log":{"source":"beat/metrics-monitoring"},"ecs.version":"1.6.0"}
```
The message is quite obscure and reported in several posts online:
- https://discuss.elastic.co/t/metricbeat-crashing-reason-slow-event-handlers-blocking-the-queue/306648/3
- https://discuss.elastic.co/t/elastic-agent-not-shipping-all-logs-from-kubernetes-cluster-errors-in-logs/325620
**Describe a specific use case for the enhancement or feature:**
We should attempt to improve the experience of the user and make those errors actionable, such as:
- If possible, return the state of the unit as `degraded` until it recovers.
- Suggest to increase the resources of the host / container in the logs
FYI @ebeahan
Contributor guide
Assessment
This issue has not been assessed yet.