GFX1100-TG200: prefill M-tiled K-quant GEMM (VT_PREFILL_TILE)
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: GFX1100-TG200
Summary
Prefill M-tiled K-quant GEMM arm that streams each weight row once and applies it to MT consecutive activation rows, cutting weight-side traffic ~MT-fold at m > 1.
Lever
VT_PREFILL_TILE=1 (default OFF). Dispatches KQuantGemmMTiledK<OutT, Fmt, 16> for m > 1 only; the m == 1 decode GEMV/coop arms are untouched. Bit-identical to the baseline KQuantGemmK dispatch.
Benchmark
A/B interleaved, 5 pairs, Qwen3.5-4B Q4_K_M, 256 tokens, temp 0, seed 0:
| A (T25 chain) | B (T25+T36) | Delta | |
|---|---|---|---|
| Median | 32.38 | 32.42 | +0.10% |
Noise in decode — T36 targets prefill (m > 1). The decode-only benchmark (m == 1) does not exercise this path.
Token identity
PASS — identical output to T25 chain.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Locate the VT_PREFILL_TILE dispatch and KQuantGemmMTiledK entry point, then compare them with the existing KQuantGemmK path. Verify the m > 1 prefill behavior, preserve the untouched m == 1 decode path, and run the reported Q4_K_M benchmark plus token-identity check to confirm completion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100