Feature Request: Add Prometheus Metrics for Monitoring Models via vLLM Router

Describe the feature

Description

It would be great to have built-in support to monitor all models running under the vLLM router through the router itself, exposing relevant metrics in Prometheus format. This would allow users to track usage, performance, and health of each model centrally without needing to access each model instance directly.

Motivation

• Centralized monitoring simplifies observability across multiple models.
• Easier integration with existing Prometheus + Grafana monitoring stacks.
• Enables tracking of metrics such as request rates, latency, error rates, token usage, and resource usage per model through the router layer.

Proposed Solution

• Extend the vLLM router to collect and expose Prometheus metrics for every model it routes requests to.
• Suggested metrics include:
  • Number of requests per model
  • Latency (request processing time) per model
  • Error counts per model
  • Number of tokens processed per model (input and output tokens)
  • Resource usage or load indicators (if feasible)
• Expose an HTTP endpoint (e.g., /metrics) on the router for Prometheus scraping.

Additional Context

This feature would improve operational visibility when running multiple models behind a vLLM router, and aligns well with common best practices for observability in microservice architectures.

Why do you need this feature?

No response

Additional context

No response

Start by locating the vLLM router entry point and its model-routing request paths, then check whether an existing metrics endpoint or Prometheus integration is present. Define the metric labels and collection points for per-model requests, latency, errors, tokens, and resource indicators before implementing the /metrics endpoint. Done means the router exposes documented Prometheus metrics that cover the agreed model-level signals.