mudler / mudler/vllm.cpp

Qwen3.5 paged attention bypasses the shared AttnBlock seam

Open
#2,923 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: BACKEND-ROCM

Gap

The numerical-characterization review for #2773 found that Qwen3.5 paged attention does not enter dense_attn::AttnBlock. The repository requires that shared seam for decode. This issue owns reconciliation of that existing route; #2773 owns characterization of the route that actually executes.

At local revision 41b6aa2560a576c28fa15f2a3ff4505305f9b850, src/vllm/model_executor/models/qwen3_5.cpp::RunDenseLayerPaged calls FullAttnBlockPaged at line 7758. FullAttnBlockPaged starts at line 5726 and calls dense_attn::WriteKvCache at line 5909. The file has no call to dense_attn::AttnBlock.

This is a source finding, not a new runtime failure. The custom route predates this characterization: git log -S'FullAttnBlockPaged' identifies the paged-forward additions f1ae0183f and 13fd1ea7f. Characterization probes must name this executing path rather than assert the mandated seam is already reached.

Scope and gates

Prepare a committed spec before implementation. Determine whether the shared seam can represent Qwen3.5 paged attention; extend it where necessary, then route the model through it without changing the pinned vLLM behavior or dtype defaults. Preserve the quantized arms.

Require a failing test through ModelRegistry::Forward, focused correctness gates, and a fresh review that deletes the production call site and proves the gate fails. Read the complete matching active-pin vLLM chain and run the identical workload before acceptance.

The active vLLM pin is e126687a9a828d513c01a07cd69f025f27d63280. Its local source objects are available. A runnable matching ROCm oracle remains PENDING in #2773. No performance result or shared-seam completion is claimed here.

Ownership

BACKEND-ROCM owns the gap. The spec .agents/specs/rocm-qwen35-08b-cpu-gfx1100-numerics.md will name this issue under ## Owed and identify the existing route explicitly. Wiring is separate from #2773's instrumentation change and needs its own reviewed spec. The exception records existing debt; it does not waive the shared-seam requirement.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with src/vllm/model_executor/models/qwen3_5.cpp, especially RunDenseLayerPaged and FullAttnBlockPaged, then read the active-pin vLLM chain at e126687a9a828d513c01a07cd69f025f27d63280. Prepare the required spec and a failing ModelRegistry::Forward test before implementation. Done means the paged route uses dense_attn::AttnBlock without changing pinned behavior or dtype defaults, preserves quantized arms, and passes the focused correctness gates.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
ai, backend
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.