agent-substrate / agent-substrate/substrate

[benchmarking] Expose host kernel PSI metrics and relabel root cgroup in cAdvisor

Aperta
#1,588 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
area/benchmarking area/observability kind/feature
Lingua principale
Go
Stelle
1.8k
Fork
316
Merge medio
2g 43m
PR unite (30g)
287

Descrizione

### Background & Problem
When benchmarking Agent Substrate under realistic agent workloads (running test suites, compilation, and git operations across concurrent actor sandboxes), diagnosing whether tail latency spikes and cold-start regressions stem from CPU throttling, memory thrashing, or disk I/O contention requires Linux kernel Pressure Stall Information (PSI).

Currently, cAdvisor in `benchmarking/monitoring.yaml` drops PSI metrics and leaves the root cgroup `id="/"` unlabeled, making it difficult to isolate host-level resource starvation from actor container pressure.

### Proposed Changes
1. Relabel the root cgroup `id="/"` to `container="node"` in the cAdvisor scrape job in `benchmarking/monitoring.yaml`.
2. Expand the metric allowlist to capture `container_pressure_{cpu,memory,io}_waiting_seconds_total`, along with network throughput and filesystem usage.

### References
- [Substrate Benchmark Telemetry & Metrics Enhancement](https://docs.google.com/document/d/1f9b1mTyxICeLZPojrxCujZ16Elh4IFo3_WmBeNCvk1c/edit?pli=1&tab=t.0)
- [Agent Substrate: Actor Density Benchmark Specs](https://docs.google.com/document/d/1sv5aBXvGOQ69iaqFxdPZ-d6tNbh5AMuwKODDyYjzYQw/edit?tab=t.0)

cc @maxsmythe @roycaihw - happy to submit a PR for this!

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.