SPEC-DFLASH2 A2-5: lift the async input-combine veto, the wave that captures the idle
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: SPEC-DFLASH2
Spec: .agents/specs/dflash2-async-spec-sampler.md, wave A2-5 — "flip the veto, and only here. The flip is one line at two sites; every wave above exists so that line is safe."
Why it is justified now, and why the arithmetic changed
The row's target was mis-stated and is now corrected. The evidence gives ours 14.914 vs vLLM 16.323 tok/s — a 9.45% throughput gap, not the 11.1% per-step-time ratio quoted earlier. Those differ because we accept MORE per step (4.358 vs 4.295 mean acceptance length), so a slightly longer step still wins on tokens/second.
| throughput gap to close | 9.45% |
| step-time reduction that achieves it at our acceptance | 8.63% |
| decomposition ceiling (idle 3.99% + non-GEMM 3.46% + GEMM 2.61%) | 10.06% |
| perfect levers | 16.581 tok/s vs 16.323 — ahead by 1.58% |
The known levers suffice. Idle is the largest single one at 3.99%, and A2-5 is the wave that captures it. Recorded in .agents/benchmark-record.md at 987b74ca6, with the five figures that did not reproduce from the evidence named there.
Every blocker is landed
- A2-1 (
e64f00560) — the draft-aware combine and its G2 instrument. - A2-2 (
7be09d9f3) — device-resident verify; the accept walk's D2H moved to a copy queue. - A2-3 (
03736b2fe) — device-resident propose; the draft buffer the combine reads. - #2920 (
18157e04a) — the async decode arm proposes, so a spec engine's first step no longer drafts nothing. - A2-4 (
4269469da) — the propose reads committed ids where they live. Thecommitted_ids_on_hostrefusal is DELETED, not bypassed:DownloadCommittedIdsruns unconditionally outside every write-back branch. This is what made the veto liftable.
What is asked
Lift the veto at both GPUModelRunner construction sites, and only that. Every wave above exists so this line is safe.
The gates this wave owes, from the spec
- G3 (GPU). TPOT and output-throughput A/B at c=1 and c=8 on the #1574 DFlash2 workload, operator-run under an
rclease, against the same binary withVT_ASYNC_RUNNER=0. Spread at c=8 is about 5.9% and at c=1 about 1.6%, so a delta under the floor is reported as under the floor and never as a ratio. - G4 is RUN: 96.01% busy node mode, 96.53% graph mode, recorded at
987b74ca6.
Hazards, all of which this row has paid for
- Reason A has been armed twice and disarmed twice. A2-3 re-armed it by passing
cu_num_logits = nullptrfrom both CUDA arms, which driveslogits_starttoquery_end - 1— the last draft slot on a verify step. Lifting the veto is exactly what makes such a defect live, and it is invisible to every token gate because verify is lossless and only acceptance moves. Check both CUDA arms pass the realcu_num_logitsbefore flipping anything. - Predicate split. A per-request refusal paired with a per-step route predicate shipped here as a silent wrong answer and survived 27 mutations because every test used
num_reqs == 1(#2710). - Gates that measure nothing. Three separate A2-3 reviews each found a guarantee no test could red.
- The emit-count discriminator is not a committed gate, because reaching the pre-repair shape needed the veto deleted. Once the veto is lifted, that discriminator becomes constructible — and it is the instrument that separates a correct combine from reason A.
What this wave may not claim
No throughput result until G3 is measured. The 10.06% is a perfect-case ceiling against our own wall, not a measured gain, and RECORDED, no floor declared is not a pass.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read .agents/specs/dflash2-async-spec-sampler.md and inspect both GPUModelRunner construction sites, then verify both CUDA arms pass the real cu_num_logits before changing the veto. Done means the veto is lifted only at those two sites and G3 A/B measurements at c=1 and c=8 are run against VT_ASYNC_RUNNER=0; do not claim a throughput result before measuring.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, performance, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100