alunduil / alunduil/blog.alunduil.com
The observability stack outweighed the bug it was built to catch
- 主要语言
- Astro
- 星标
- 1
- 派生
- 1
- 平均合并
- 6 小时 34 分钟
- 30 天内合并 PR
- 118
描述
## Spark
A long-running workstation-freeze investigation closed this week by deleting the local Alloy/Prometheus/Loki/Tempo/Grafana stack and shipping `sar` output to Grafana Cloud instead.
## Why it could be interesting
The instrumentation built to observe an intermittent freeze was heavy enough to be a suspect in it, so the resolution was to shrink the observer rather than sharpen the observation. The claim: for resource-contention bugs on a single machine, local telemetry infrastructure is a confound before it's a tool, and the right move is to push collection off the box under investigation.
## Open questions
- [ ] Was the freeze diagnosed, or outlived? The post needs a real answer — no claiming a fix that wasn't verified.
- [ ] What did the local stack actually cost in the resources under investigation (memory, IO)? Measured or inferred?
- [ ] Is the Grafana Cloud shipper sufficient to catch a recurrence, and has that been tested?
## Source material
- https://github.com/alunduil/alunduil-chezmoi/issues/404 (closed) and https://github.com/alunduil/alunduil-chezmoi/pull/662 — the swap
- https://github.com/alunduil/alunduil-chezmoi/issues/351 (closed) — retry-on-EAGAIN zellij built to watch the mechanism live
- https://github.com/alunduil/alunduil-chezmoi/issues/347 (closed) — timestamped launch markers
- https://github.com/alunduil/alunduil-chezmoi/issues/356 (closed) — hook cut from ~8 processes to one jq
- https://github.com/alunduil/alunduil-chezmoi/issues/354 (closed) and https://github.com/alunduil/alunduil-chezmoi/pull/671 — least-privilege systemd run-as
- https://github.com/alunduil/alunduil-chezmoi/issues/675 — monitoring moves to system units
贡献指南
这个仓库没有索引到贡献指南
评估
这个 Issue 还没有评估数据。