fix(BACKEND-ROCM): bound HRX decode writes to each KV group
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: BACKEND-ROCM
The native HRX cooperative split-K attention reducer permits multiple KV groups to write the same query-head output. At AMD-Ecosystem/llama.cpp pin 6319038132ed12f968ea68f37753f705da830ea8, flash_attention_decode_split_f32_f16_wmma.loom:578 tests each global head against the total query-head count. It must test the local query row against the heads owned by that KV group.
For Qwen3-0.6B with 16 query heads and 8 KV heads, KV0 can write heads 0–7 and KV1 can write heads 2–9. The cooperative path begins at KV capacity 257. The unchanged direct reducer owns exactly two heads per KV group.
The #3083 sequential backend fixture fails on its first decode at position 256: maximum absolute error 0.056254 exceeds the unchanged upstream flash-attention bound of 0.05. Shared weights are byte-identical and the cache-only control passes. The operator's whole-model boundary test matches HIP for all 128 output IDs after prompts of length 16 and 128. Prompt 240 first diverges at output index 17; prompt 256 first diverges at index 1. Both failures begin at decode position 256.
The #3083 implementer owns the bounded validity-predicate repair in the same flow. Preserve dispatch, arithmetic, thresholds, the #3081 provider, and upstream pins. Retain the numerical and whole-model red evidence, then require green and a fresh mutation review. Parent: #3080. Spec: .agents/specs/rocm-hrx-prefill-repair-3083.md.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with flash_attention_decode_split_f32_f16_wmma.loom at line 578 and read .agents/specs/rocm-hrx-prefill-repair-3083.md alongside the #3083 sequential backend fixture. Verify the local query-row validity bounds each KV group without changing dispatch, arithmetic, thresholds, provider #3081, or upstream pins. Done means the numerical and whole-model checks are green and a fresh mutation review is complete.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 28/100