vllm-project / vllm-project/vllm
[Doc]: metrics page is missing the spec-decode counters and cache_config_info
- Dominant language
- Python
- Stars
- 91.8k
- Forks
- 22.2k
- PR merge metrics
- PR metrics pending
Description
### 📚 The doc issue
The metric tables on https://docs.vllm.ai/en/latest/usage/metrics/ come from `docs/mkdocs/gen_files/generate_metrics.py`. That script only picks up a metric when `name=` is a string literal. Two files don't do that:
- Since #45163, `vllm/v1/spec_decode/metrics.py` creates its counters in a loop over `counter_specs`, calling `self._counter_cls(name=name, ...)`.
- `vllm/v1/metrics/loggers.py` builds `vllm:cache_config_info` through a local variable.
As a result, these metrics are missing from the page:
- `vllm:spec_decode_num_drafts`, `vllm:spec_decode_num_draft_tokens`, `vllm:spec_decode_num_accepted_tokens`
- `vllm:diffusion_num_denoising_steps`, `vllm:diffusion_num_canvas_positions`, `vllm:diffusion_num_committed_tokens`
- `vllm:cache_config_info`
The Speculative Decoding table lists only `vllm:spec_decode_num_accepted_tokens_per_pos`. Meanwhile `features/speculative_decoding/acceptance_metrics.md` tells users to compute acceptance from `vllm:spec_decode_num_drafts_total` and `vllm:spec_decode_num_draft_tokens_total`, and neither of those is on the page.
### Suggest a potential alternative/fix
- Let the extractor resolve `name=` when it's a local variable, either assigned a string or unpacked from a literal table.
- Have the generator fail the docs build when a `vllm:*` name in a scanned file isn't extracted, so this can't regress silently again.
I have a fix ready and will open a PR.
### Before submitting a new issue...
- [x] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the [documentation page](https://docs.vllm.ai/en/latest/), which can answer lots of frequently asked questions.
Contributor guide
Assessment
This issue has not been assessed yet.