defer(SPEC-ACCEPT-VARIANTS): the draft-logits cache dtype change is inert, and it makes a recorded layout stale
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: SPEC-ACCEPT-VARIANTS
Found by wave PORTQ-2 re-deriving PORT-NOW entry 80 of 5559679229..e126687a9a
(#2646). Upstream:
81bc196913 vllm#50910, "[Model Runner V2] Cache draft logits in model's LM head
dtype".
Deferred work with a named gate, filed so the person who lands the gate meets
it.
The gate: probabilistic draft sampling is refused by name
src/vllm/config/speculative.cpp:489-496 refuses any draft_sample_method other
than "greedy", and its message names this row:
"This engine drafts greedy only ... Probabilistic draft sampling is owed by row
SPEC-ACCEPT-VARIANTS"
The whole commit is about the draft_logits CACHE and the temperature algebra
around it. That cache exists upstream only when the draft samples
probabilistically, and it is consumed only by the stochastic rejection sampler.
Neither is reachable here: _cache_draft_logits is listed as step 6 and marked
"(probabilistic only; owed)" at
include/vllm/v1/worker/gpu/spec_decode/dflash2/speculator.h:13, and the
stochastic verify is deferred at
include/vllm/v1/spec_decode/rejection_sampler.h:53-57.
CORRECTION: this commit does NOT make our recorded layout stale
An earlier version of this issue said it did. That was wrong, and acting on it
would have corrupted a correct record. It claimed 81bc196913 changes the
draft-logits cache to "the LM-head dtype holding pre-temperature logits", making
.agents/specs/dflash2-spec-decode.md's recorded (torch.float32, -inf) stale
on the pin advance. Verified at the advance target e126687a9a:
- the commit's dtype hunk edits
DraftModelSpeculatorin the BASE
vllm/v1/worker/gpu/spec_decode/speculator.py, whosedraft_logits_spec
returnsmodel_config.head_dtype, 0.0at:332-337; - DFlash2 has its own
draft_logits_spec, and it still returns
torch.float32, -float("inf")at
vllm/v1/worker/gpu/spec_decode/dflash2/speculator.py:133-137, with the reason
in a comment beside it; - DFlash2 writes that cache with its own
_cache_draft_logits_kernel
(dflash2/speculator.py:79-105, launched at:167), never through
gumbel_sample'slogits_cache, which is the parameter this commit renames; - the commit's file list is
dspark/speculator.py,gumbel.py,
rejection_sampler_utils.pyand the basespeculator.py— not
dflash2/speculator.py.
So neither half reaches DFlash2. Our record at
include/vllm/v1/worker/gpu/spec_decode/dflash2/speculator.h:31,35-36,
include/vt/ops.h:1692 and .agents/specs/dflash2-spec-decode.md:852,1627 is
correct at the advance target. Nothing is owed here, and a pin-advancing wave
should leave those lines alone.
The inert disposition is unaffected and holds on its own ground, stated
above: probabilistic draft sampling is refused by name, so the cache this commit
re-dtypes cannot exist in any configuration this tree builds. What changed is the
reasoning attached to it, not the label.
Found by a fresh review of #2675; the wave's report carries the same retraction
at .agents/sync/2026-09-03-portq2.md §9.1.
Nothing was executed for this finding: no build, no test, no GPU.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading src/vllm/config/speculative.cpp:489-496 and the DFlash2 speculator files cited in the issue, then compare the upstream commit's file list. No patch or test is requested; the work is complete when the inert disposition and the recorded DFlash2 layouts remain verified and unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- backend, machine-learning
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100