mudler / mudler/vllm.cpp

Public docs drift from shipped CLI, ABI v17, endpoints, registry counts, and benchmark scope

Open
#342 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Problem

The user-facing README and usage guide disagree with current source and current keyed public records:

  • README.md invokes build/examples/server, but the CMake target outputs build/examples/vllm-server.
  • README and docs/USAGE.md describe ABI v10 with 19 exports, while include/vllm.h declares ABI v17 and the current public surface.
  • docs/USAGE.md omits the conditional /v1/embeddings and /v1/audio/transcriptions routes implemented in api_server.cpp.
  • README says 25+, 28, and 30 architectures in different places, while the registry-bound docs/FEATURES.md checker proves 35 registered and 27 gated.
  • README says every architecture is token-gated, although registered scaffold/oracle-blocked rows are explicitly documented.
  • README's global throughput language conflates the passing unsloth@890bdef7 27B grid with the current NVIDIA ModelOpt 27B and 35B grids that remain speed-pending.

Scope

Create one source-grounded documentation repair:

  1. Correct the quickstart/server executable path.
  2. Update ABI wording and the ABI evolution table through v17 from include/vllm.h.
  3. Add the shipped conditional embedding and transcription endpoints to docs/USAGE.md.
  4. Make README registry counts and gate wording agree with docs/FEATURES.md.
  5. Scope throughput claims to the exact checkpoint and keep current ModelOpt/35B gaps honest.
  6. Apply a human-language pass without adding new claims or benchmark numbers.

Do not compact docs/STATUS.md or docs/BENCHMARKS.md in this issue. Do not change code, checker semantics, lifecycle state, or accepted benchmark values.

CPU-only acceptance

  • python3 scripts/check-readme-structure.py
  • python3 scripts/check-public-doc-tables.py
  • python3 scripts/check-supported-models.py
  • python3 scripts/check-surface-coverage.py
  • Configure/build the CPU examples and prove build/examples/vllm-server --help.
  • Stale-string scan finds no old ABI/count/path claims.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with README.md and docs/USAGE.md, then verify ABI declarations in include/vllm.h, routes in api_server.cpp, and registry counts and gate wording in docs/FEATURES.md. Run the four listed Python checkers before configuring CPU examples and testing build/examples/vllm-server --help. Done means all checks pass and stale ABI, count, and path claims are absent without changing code, checker semantics, lifecycle state, or benchmark values.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, cpp, python
Domain
api, build-system, documentation
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.