perf(SPEC-GDN-SEGMENTS): the GDN builder computes context_lens on decode-only batches that never read it
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: SPEC-GDN-SEGMENTS
Found by wave PORTQ-4 (#2680)
re-deriving PORT-NOW entry 150 of 5559679229..e126687a9a, upstream
10b7766a90 vllm#51913
"[Attention] Move context_lens_tensor compute into GDN prefill path".
This is the one entry in the tranche whose recorded PORT-NOW line made a claim
about this tree and was right. The line says "vllm.cpp mirrors gdn_attn and this
elides that per-step computation on decode-only batches"; it checks out.
What is wrong
GdnMetadataBuilder::build computes context_lens unconditionally at the top:
src/vllm/v1/attention/backends/gdn_attn.cpp:97-104
// context_lens = num_computed_tokens = seq_lens - query_lens
const std::vector<int32_t> query_lens = m.naive_query_lens();
std::vector<int32_t> context_lens(static_cast<size_t>(m.num_reqs), 0);
for (int r = 0; r < m.num_reqs; ++r) { ... }
Its only two consumers, at :316 and :326, are both inside the
if (num_prefills > 0) block opened at :315 — the exact block upstream moves the
computation into.
The header already states the invariant the source does not honour:
include/vllm/v1/attention/backends/gdn_attn.h:106-110 — "Upstream ONLY populates
it when num_prefills > 0 (the decode path never needs it — a decode request always
continues an existing sequence)".
Size, stated honestly
Move the query_lens fetch and the context_lens loop inside the
num_prefills > 0 block. ~6 lines, one file, plus a decode-only-batch case in
tests/vllm/v1/attention/test_gdn_metadata_builder.cpp.
The win is small. The elided work is one host loop over num_reqs per step,
not per layer. This is a real divergence from upstream and a cheap one; it is filed
so it is not lost, not because it is urgent. It may reasonably ride the next
SPEC-GDN-SEGMENTS change rather than take its own pull request — that row's scope
line already names both files as ones it owns.
Verification state
Nothing was executed — no build, no test run, no GPU, no lease. Both the
unconditional loop and the guarded consumers were printed from the tree at
a700e8da6 and re-read by the wave operator. gdn_attn.py is present at
5559679229 and the pre-fix placement is the pin's, so this is a genuine post-pin
forward-port. No prior triage: git log --all --grep 10b7766a90 is empty.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in src/vllm/v1/attention/backends/gdn_attn.cpp at GdnMetadataBuilder::build and read the invariant in include/vllm/v1/attention/backends/gdn_attn.h. Move the context_lens computation into the existing num_prefills > 0 block, then add or update the decode-only case in tests/vllm/v1/attention/test_gdn_metadata_builder.cpp. Run that test to verify decode-only batches avoid the computation while prefill behavior remains correct.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, performance
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 84/100