GFX1100-TG200: fuse silu-mul with Q8_K quant epilogue (VT_SILU_QUANT_FUSED)
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: GFX1100-TG200
Summary
Fuse the MLP silu(gate)*up activation with the Q8_K quantization epilogue on ROCm, eliminating the standalone QuantizeQ8KKWarpCoop launch for the ffn_down GEMV's activation input.
Lever
VT_SILU_QUANT_FUSED=1 (default OFF). When enabled, SiluMulQuantQ8KKWarpCoop produces both the bf16 activation and the Q8_K scratch in a single kernel, deleting the separate quant launch (~8.9us).
Benchmark
A/B interleaved, 5 pairs, Qwen3.5-4B Q4_K_M, 256 tokens, temp 0, seed 0:
| A (T25 chain) | B (T25+SiluQuant) | Delta | |
|---|---|---|---|
| Median | 32.41 | 32.39 | -0.08% |
Noise — the fusion saves a launch but the benchmark is decode-dominated and the quant launch is amortized.
Token identity
PASS — identical output to T25 chain on The capital of France is with --max-tokens 32 --temperature 0 --seed 0.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Locate the ROCm implementations of SiluMulQuantQ8KKWarpCoop and QuantizeQ8KKWarpCoop, then trace the ffn_down activation path and the VT_SILU_QUANT_FUSED lever. Verify that the fused path produces both bf16 activation and Q8_K scratch without the standalone quantization launch, while preserving token identity and the reported benchmark behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100