vllm-project / vllm-project/aibrix
[ModelClaim] Add operational metrics for pooled runtimes and policy actions
@FAUST-BENCHOU is already working on this.
Since Jul 18, 2026.
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 697
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 104
Description
Feature Description and Motivation
ModelClaim exposes basic desired/ready/activation metrics and runtime KV/HBM gauges, but operators cannot currently tell whether an engine is repeatedly restarting, was re-adopted after an agent restart, exhausted its restart budget, or whether the pool KV policy applied or skipped a decision.
ModelClaim is still experimental, so this issue is limited to additive observability.
Use Case
When several model engines share one GPU, an operator needs to distinguish a healthy dense pool from a pool that is silently converging through engine restarts or repeatedly failing policy actions. The signals must support alerts and post-incident diagnosis without requiring runtime snapshot or controller log inspection.
Proposed Solution
Add low-cardinality Prometheus metrics covering:
- engine restart attempts and restart-budget exhaustion;
- runtime agent re-adoption outcomes;
- engine state counts by phase (
active,sleeping,restarting,failed); - pool-policy evaluation and action outcomes (
applied,skipped,failed) with a bounded reason label; - requested and successfully applied KV limits per model;
- sleep and wake operation count and duration;
- duration for which a ModelClaim has no ready engine.
Keep labels bounded to namespace, model, pod/pool where necessary, action/result, and an enumerated reason. Do not expose free-form errors, operation IDs, PIDs, or claim UIDs as metric labels. Detailed errors should remain in logs and Kubernetes Events.
Add focused unit tests for registration, values, cleanup of deleted gauge series, and bounded label values. Document metric names and semantics alongside the ModelClaim runbook.
Acceptance Criteria
- Metrics are exported from the existing controller-manager or runtime
/metricsendpoints. - A terminal engine failure, successful re-adoption, successful/failed KV update, and sleep/wake operation are observable without reading logs.
- Policy skip reasons are from a fixed enum and cannot introduce unbounded cardinality.
- Existing ModelClaim APIs and reconciliation behavior remain unchanged.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.