mudler / mudler/vllm.cpp

Jump-forward decoding ships a server flag, an ABI field and a FEATURES tick, and its capability function has no caller

Open
#2,387 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

What

The capability terminates in a bool nobody reads.

$ git grep -n "DrainForcedTokens" -- src include tests examples
include/vllm/v1/structured_output/jump_forward.h:70   (decl)
src/vllm/v1/structured_output/jump_forward.cpp:25     (def)
tests/vllm/v1/structured_output/test_jump_forward.cpp:39,185,288,294

$ git grep -n "jump_forward_enabled()" -- src include tests
include/vllm/entrypoints/model_loader.h:581   (the accessor itself)
tests/capi/test_capi.cpp:1337,1345,1353

DrainForcedTokens has no production caller, and the accessor that would gate it is read only by tests.

Everything up to the latch IS wired, which is what makes it convincing

  • src/vllm/entrypoints/openai/server_main.cpp:633-645 parses --enable-jump-forward and hard-errors if passed twice — a strong "this flag is real" signal
  • :1293 assigns it
  • src/capi/vllm_c.cpp:712-727 validates the ABI field
  • model_loader.cpp:2022-2023 resolves it into jump_forward_enabled_

Then nothing consumes it.

What the user-facing surfaces promise

  • include/vllm.h:77,551-560 — ABI v10 added vllm_model_params.enable_jump_forward as a public tri-state field
  • docs/reference/server.md:202| --enable-jump-forward | off | Jump-forward decoding for structured output (token-unique subset) |
  • docs/FEATURES.md:278| Jump-forward decoding | ✅ opt-in |
  • docs/SGLANG-COMPAT.md:155 shows a copy-pasteable server --model ... --enable-jump-forward

docs/ENVIRONMENT.md:120 is honest — "Currently drives only the standalone driver (DrainForcedTokens) ... Off by default until the production scheduler splice ... lands" — but server.md and FEATURES.md are the two a user reads first, and neither says so.

Fix, either direction

Land the scheduler splice, or correct server.md and FEATURES.md to match ENVIRONMENT.md's honesty. What must not persist is a documented, validated, ABI-versioned flag that provably does nothing.

Row: ENG-STRUCTURED-OUTPUT

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the flag through server_main.cpp, vllm_c.cpp, model_loader.cpp, and the jump-forward tests, then compare the user-facing claims in docs/reference/server.md and docs/FEATURES.md with docs/ENVIRONMENT.md. Determine whether the scheduler integration or documentation correction is the appropriate scope; done means the flag's documented behavior matches a production caller or clearly states its current limitation, with relevant tests passing.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend, documentation
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.