vllm-project / vllm-project/aibrix

[ModelClaim] Add a Grafana dashboard for high-density runtime pools

Open
#2,457 5 comments 0 reactions 0 assignees View on GitHub
area/observation good first issue help wanted kind/enhancement
Dominant language
Go
Stars
5.1k
Forks
697
Avg merge
1d 19h
Merged PRs (30d)
104

Description

### Feature Description and Motivation

ModelClaim needs a dedicated Grafana dashboard for operating experimental high-density GPU runtime pools. Existing dashboards do not present ModelClaim lifecycle, per-engine runtime health, or kvcached allocation in one view.

This issue is intentionally separate from adding missing Prometheus metrics. The dashboard should use the metrics available on the target branch and may be completed after the operational-metrics issue lands.

### Use Case

An operator should be able to open one dashboard and answer:

- Which ModelClaims are active, activating, sleeping, or failed?
- Which pool or Pod hosts each model and how dense is the pool?
- Are engines restarting or exhausting restart budgets?
- How are KV capacity and current KV use distributed among co-resident models?
- Is the pool policy applying changes, skipping unsafe plans, or failing actions?
- How much HBM headroom remains and which engines have the largest observed peak?

### Proposed Solution

Add a dashboard JSON under `observability/grafana/` with:

- namespace, pool, Pod, and model variables;
- ModelClaim desired/ready and lifecycle overview;
- resident model count per runtime Pod;
- per-model KV used, KV limit/capacity, and HBM peak panels;
- engine restart/failure and agent re-adoption panels;
- policy action/skip/failure panels;
- sleep/wake activity and no-ready duration panels;
- links or annotations that make Kubernetes Events/log investigation straightforward.

Follow the repository's existing dashboard conventions. Avoid panels that depend on free-form/high-cardinality labels. Include a short README section describing required metrics and how to import or provision the dashboard.

### Acceptance Criteria

- Dashboard JSON is checked in under `observability/grafana/` and is valid JSON.
- Empty/no-data states render cleanly.
- Variables allow a user to narrow from namespace/pool to Pod/model.
- Panels do not use metrics outside the documented ModelClaim metric contract.
- The dashboard works with multiple warm pools and multiple co-resident models.

Contributor guide

Open the contributing guide

Research direction

Inspect existing dashboard JSON files under observability/grafana/ and the repository's dashboard conventions first, then review the documented ModelClaim metric contract on the target branch. Validate the new JSON and check that variables, no-data states, panels, and README import or provisioning guidance satisfy the acceptance criteria without relying on undocumented metrics.

Written by the indexing model from the issue text.

Assessment

Tech stack
grafana
Domain
observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.