Wire production AsyncLLM requests into Prometheus /metrics
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Problem
The shipped vllm-server --enable-metrics exposes the Prometheus catalog, but production requests run through AsyncLLM while the logger is attached only to the unused synchronous LLMEngine. Live scrapes therefore leave num_requests_running, token counters, request-success counters, and timing histograms at zero even while requests complete.
Reproduction
Run four concurrent /v1/chat/completions requests against a server started with --enable-metrics and scrape /metrics at 100 ms cadence. Requests complete correctly, but all nine aligned samples report num_requests_running=0, num_requests_waiting=0, with token and success counters unchanged.
Scope
Complete W6 in .agents/specs/prometheus-metrics.md:
- attach the existing
PrometheusStatLoggerto productionAsyncLLM; - build and record
IterationStatsin the output-handler path; - make concurrent
Record()/Expose()safe; - preserve the null-logger no-stats path;
- add RED-first async metric and concurrent-scrape tests;
- gate affected async/sync/Prometheus/server targets.
No model kernels, sampling, scheduler policy, or Hermes client changes.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read .agents/specs/prometheus-metrics.md and trace the production AsyncLLM output-handler path alongside PrometheusStatLogger and IterationStats. Add the async metric recording and concurrency coverage described there, preserving the null-logger path, then run the affected async, sync, Prometheus, and server targets to verify RED metrics and concurrent scrapes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, prometheus
- Domain
- backend, observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100