Require distinct Q and K projections in native HRX Qwen fusion
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: BACKEND-ROCM
The pinned AMD llama.cpp Qwen attention matcher accepts a K projection as both the query and key chain. When a complete graph visits K before Q, it emits one partial Qwen postprocess and leaves the real query to generic RMSNorm and RoPE. match_qwen_attention_postprocess in ggml/src/ggml-hrx/dispatch_registration/qwen/dispatch-qwen-attention-postprocess.cpp:623 does not require distinct query and key projection nodes.
The #3083 implementer owns the in-flow scheduling correction. The reservation helper already rejects this ambiguity, but the actual matcher must enforce the same complete-region invariant. Preserve the original matching and traversal defaults, numerical tolerances, and the complete Qwen fusion.
The CPU regression enters the public dispatch scheduler with BF16 Q16/KV8, token count1, and K-before-Q graph order. It fails at the assertion that a complete Qwen postprocess leaves no generic RMSNorm: exit -6, test ELF 1fe0b0bbf1a22a84fb652816ad943e678e415d167da8bfb9383c3db9afb613a8, unchanged backend fb2301563318df2253bc681272247a2a5587a42921a9b376b89fa103de70b2a9. Evidence: build-hrx-prefill-repair/evidence/key-first-scheduler-red.{json,log} in the #3083 worktree.
The #3083 spec will record this narrow matcher obligation before its implementation. A separate sequential GPU regression currently rejects NaN in the first decode K cache. This issue does not yet attribute that numerical failure to this matcher ambiguity; the operator will rerun it after the scoped correction.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in ggml/src/ggml-hrx/dispatch_registration/qwen/dispatch-qwen-attention-postprocess.cpp at match_qwen_attention_postprocess around line 623, then review the key-first scheduler evidence in build-hrx-prefill-repair/evidence/key-first-scheduler-red.{json,log}. Verify the matcher requires distinct query and key projection nodes while preserving the existing defaults and complete fusion, then rerun the CPU regression with BF16 Q16/KV8 and token count 1 to confirm no generic RMSNorm remains.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- ai-infra-agents, backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100