port(ENG-SCHED-CORE): no drafting-slot reservation, so a spec-decode step schedules more input tokens than vLLM does
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: ENG-SCHED-CORE
Found by wave PORTQ-4 (#2680)
re-deriving PORT-NOW entries 129 and 156 of 5559679229..e126687a9a — upstream
0914ed2e81 vllm#51725 and
2ac1f683f1 vllm#51256.
They are filed together because they are one unit of work. [156] adds the
property and [129] is its only consumer; porting [156] alone lands dead code, and
porting them apart installs a subtraction that [129] then removes.
What is wrong, and why the queue understates it
Under speculative decoding this tree schedules more input tokens per step than
vLLM does, because it has no drafting-slot reservation at all.
include/vllm/config/scheduler.h:166-168—ResolvedMaxNumScheduledTokens()is a
baremax_num_scheduled_tokens.value_or(max_num_batched_tokens)with no drafting
delta.new_slotsandinput_budgetare 0 hits acrosssrc/,include/and
tests/; the positive controltoken_budgetreturns 26 hits insrc/and
include/, so the zeros are real.- Every site upstream's post-commit
input_budgettouches has atoken_budget
twin here, which gives the port a clean mirror target:
src/vllm/v1/core/sched/scheduler.cpp:640,653,687,748,791,855,907,910,973,1008.
This is a pre-pin hole, not a two-commit lag. Upstream already carried
max_num_new_slots_for_drafting and its subtraction at 5559679229:
git show 5559679229:vllm/config/speculative.py:1292-1305 for the property and
5559679229:vllm/config/vllm.py:1694-1743,1704,1734 for its two consumers. A queue
derived from a commit range structurally cannot see this shape.
The divergence is already recorded in a committed golden
tests/parity/goldens/dflash_27b/d0_blocked_traceback.txt:28 captures the pinned
oracle logging, on this project's own DFlash 27B gate configuration:
max_num_scheduled_tokens is set to 4036 based on the speculative decoding settings.
while this tree schedules the full max_num_batched_tokens. This is a reading of
a committed file, not a measurement. Nothing was run. But it means any DFlash
throughput comparison against that oracle has been comparing unequal per-step token
budgets, and whoever ports this should re-check the affected benchmark rows.
What to build
SpeculativeConfig::MaxNumNewSlotsForDrafting()in
include/vllm/config/speculative.h, implementing the post-2ac1f683f1
table: dflash to K, parallel + draft_model to K, parallel to K-1, draft_model to
1, else 0. It belongs besideNumLookaheadTokens()at:520-529, which already
encodes the DFlashk + 1versus DSparkkasymmetry. Every input exists:
speculative.h:100parallel_drafting(set:158dflash,:346dspark),
:467,:479-482,:487,:491,:498. ~20 lines.- A per-step
input_budgetinScheduler::Schedule, charged
max_num_new_slots_for_draftingper scheduled request. ~15-20 lines plus one
stored member nearinclude/vllm/v1/core/sched/scheduler.h:328. - The post-commit refusal when
max_num_batched_tokens <= scheduled_token_delta.
~10 lines. - The ported upstream table test —
tests/test_config.pyadds eight cases
(eagle3/0, p-eagle/7, dflash/8, dspark/7, mtp/0, ngram/0, draft-model/1, pard/8).
Total roughly 60-80 lines of product code plus tests.
Owed
The PARD arm (parallel_drafting and uses_draft_model together) may not be
constructible here — src/vllm/v1/spec_decode/draft_model_proposer.cpp never sets
parallel_drafting — so that row of the table would land inert. It is part of the
ported artefact and should be named rather than dropped. SPEC-DRAFT-MODEL
(.agents/engine-matrix.md:194) owns that arm; SPEC-DFLASH (:179) and
SPEC-DSPARK (:181) own the polarity the table encodes.
What was not determined
Whether the tree ever overflows today. Upstream's reservation protects the
worker's token capacity when the drafter inserts mask queries; whether this tree's
DFlash runner has an independent cap that makes the overflow unreachable was not
settled statically and needs a run.
Nothing in this wave was executed — no build, no test run, no GPU, no lease.
Every path:line was printed from the tree at a700e8da6; the scheduler and
speculative-config anchors were re-read by the wave operator.
A correction the record owes
.agents/sync/2026-09-01-cdefd9d.md:1985-1990 reaches the same conclusion and is
corroborated here, but cites include/vllm/config/scheduler.h:165-167. The
resolver's body is at :166-168. Off by one, substance correct.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with SpeculativeConfig::MaxNumNewSlotsForDrafting() beside NumLookaheadTokens() in include/vllm/config/speculative.h, then trace Scheduler::Schedule and the scheduler state near include/vllm/v1/core/sched/scheduler.h:328. Compare the listed scheduler.cpp sites with the upstream behavior, add the eight cases to tests/test_config.py, and run the relevant tests; done means drafting reservations and refusal behavior are covered without dropping the inert PARD row.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- ai, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 48/100