vllm-project / vllm-project/production-stack
feature: Design a Dedicated Metrics Module for Engine/Request Metrics
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.6k
- Forks
- 503
- Avg merge
- 4d 17h
- Merged PRs (30d)
- 8
Description
Describe the feature
Currently, metrics related to engine/request in the production-stack are collected within the router——a responsibility that should ideally belong to a dedicated observability layer, This architecture leads to:
- Violation of separation of concerns: Router's core function is routing, not metrics management
- Duplicate collection overhead: Both HPA and observability systems need to scrape metrics from router, creating unnecessary 2x load
- Tight coupling: Metrics format changes require router modifications
Implementing centralized metrics storage would:
- Enable single-point collection for all consumers (cluster obser #474, router scheduling, HPA #238)
- Reduce router CPU/memory pressure by offloading metrics work
- Standardize metrics access across subsystems
Why do you need this feature?
Design a dedicated metrics module/database (similar to etcd/redis) that would:
- Collect and store all engine/request metrics centrally
- Provide a unified interface for metrics querying
- Support both real-time and historical metrics analysis
Additional context
Do you think a separate metrics module is necessary? I'm still learning about system work, so I'd be grateful for your corrections or suggestions!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files, tests, or entry points are named. Start by locating the router's current engine/request metric collection and reading related issues #474 and #238; the issue does not define a concrete implementation or an unambiguous definition of done.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100