port(SPEC-DRAFT-MODEL): async_scheduling_compatible refuses draft_model, which the pin advance makes wrong
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: SPEC-DRAFT-MODEL
Found by #2647 (PORT-NOW wave
PORTQ-3, tranche 81-120). Nothing was executed: this is a static reading of the
tree at e24ec8bfd against upstream c0202c5603 (vllm#48341).
The shape of this one is unusual, and it is the reason to file it
This tree is correct at the pin and becomes wrong the moment the pin advances.
It is the only entry in its tranche with that shape, and the tree already says so
in a comment.
include/vllm/config/speculative.h:514 is:
bool async_scheduling_compatible() const { return use_eagle(); }
and the comment directly above it, at :510-513, records the reconciliation by
name:
"draft_model"is refused AT THE PIN (b389ac2946later allows it —
vllm/config/vllm.py:1291there — but the pin governs; reconcile when the pin
advances past that commit).
c0202c5603 is the commit that does it. Its whole product delta is one conjunct
in vllm/config/vllm.py:
+ and self.speculative_config.method != "draft_model"
added to the branch that warns and disables async scheduling. c0202c5603 is an
ancestor of b389ac2946 (verified with git merge-base --is-ancestor), so the
comment's "that commit" resolves to this one or to an ancestor of it, and it is
inside the 5559679229..e126687a9a window the pin advance covers.
What to change
return use_eagle() || uses_draft_model();, plus the comment. uses_draft_model()
already exists at include/vllm/config/speculative.h:487 and "draft_model" is
an accepted --speculative-config method (:536). The predicate is reachable:
src/vllm/entrypoints/model_loader.cpp:2216-2217 consumes it to force async
scheduling off.
Size
~1 product line and its comment, plus a config-level test that a draft_model
speculative config no longer disables async scheduling. The test is the point:
without it the line is a silent default change.
Ordering
Do not land this before the pin advances past c0202c5603. Landing it early
makes the tree disagree with its own pin, which is the failure the comment at
:510-513 was written to prevent. It belongs in the pin-advance change, not
ahead of it.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at include/vllm/config/speculative.h:487 and :510-514, then trace src/vllm/entrypoints/model_loader.cpp:2216-2217 to understand the predicate's effect. After the pin advances past c0202c5603, update the predicate and comment, and add a config-level test showing that draft_model no longer disables async scheduling. Do not land this before that pin advance.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 58/100