vllm-project / vllm-project/aibrix
[ModelClaim] Add a Grafana dashboard for high-density runtime pools
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 697
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 104
Description
### Feature Description and Motivation
ModelClaim needs a dedicated Grafana dashboard for operating experimental high-density GPU runtime pools. Existing dashboards do not present ModelClaim lifecycle, per-engine runtime health, or kvcached allocation in one view.
This issue is intentionally separate from adding missing Prometheus metrics. The dashboard should use the metrics available on the target branch and may be completed after the operational-metrics issue lands.
### Use Case
An operator should be able to open one dashboard and answer:
- Which ModelClaims are active, activating, sleeping, or failed?
- Which pool or Pod hosts each model and how dense is the pool?
- Are engines restarting or exhausting restart budgets?
- How are KV capacity and current KV use distributed among co-resident models?
- Is the pool policy applying changes, skipping unsafe plans, or failing actions?
- How much HBM headroom remains and which engines have the largest observed peak?
### Proposed Solution
Add a dashboard JSON under `observability/grafana/` with:
- namespace, pool, Pod, and model variables;
- ModelClaim desired/ready and lifecycle overview;
- resident model count per runtime Pod;
- per-model KV used, KV limit/capacity, and HBM peak panels;
- engine restart/failure and agent re-adoption panels;
- policy action/skip/failure panels;
- sleep/wake activity and no-ready duration panels;
- links or annotations that make Kubernetes Events/log investigation straightforward.
Follow the repository's existing dashboard conventions. Avoid panels that depend on free-form/high-cardinality labels. Include a short README section describing required metrics and how to import or provision the dashboard.
### Acceptance Criteria
- Dashboard JSON is checked in under `observability/grafana/` and is valid JSON.
- Empty/no-data states render cleanly.
- Variables allow a user to narrow from namespace/pool to Pod/model.
- Panels do not use metrics outside the documented ModelClaim metric contract.
- The dashboard works with multiple warm pools and multiple co-resident models.
Contributor guide
Research direction
Inspect existing dashboard JSON files under observability/grafana/ and the repository's dashboard conventions first, then review the documented ModelClaim metric contract on the target branch. Validate the new JSON and check that variables, no-data states, panels, and README import or provisioning guidance satisfy the acceptance criteria without relying on undocumented metrics.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- grafana
- Domain
- observability
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100