[AMDGPU] Packed-integer codegen performance gap vs RADV/ACO on gfx1151 (3.2x slower)
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
LLVM AMDGPU backend generates significantly slower code than RADV/ACO for packed-integer operations on gfx1151, despite emitting identical instructions. This is **Issue #3** (LLVM codegen) split from parent issue [ROCm/ROCm#6409.](https://github.com/ROCm/legacy-rocm-build/issues/6409).
**Performance Gap:**
- **gfx1151 (Strix Halo):** Vulkan 3.05x-3.20x faster than HIP
- **gfx1100 (W7900):** Vulkan 1.05x-1.13x faster than HIP (minor gap)
## Evidence
Serialized workgroup-64 examples on gfx1151:
| Variant | HIP GPU time (μs) | Vulkan GPU time (μs) | Speedup | HIP dot4 count | RADV dot4 count |
|---------|-------------------|----------------------|---------|----------------|-----------------|
| q8 signed | 3581.117 | 1121.940 | **3.192x** | 16 | 16 |
| q4 unsigned × signed-q8 | 3575.964 | 1119.920 | **3.193x** | 16 | 16 |
| q6 zero-corrected | 3429.718 | 1122.946 | **3.054x** | 32 | 32 |
| **scalar q4 dequant (NO dot4)** | 3661.869 | 1142.739 | **3.204x** | **0** | **0** |
**Key observation:** Both backends emit the expected dot4 instructions. The scalar control emits **zero dot4 instructions** on both sides, yet gfx1151 still shows **3.204x** gap. This gap is **not** due to missing dot4 lowering—it's a **codegen quality issue** (scheduling, waits, address generation, occupancy).
## Architecture Dependency
- **gfx1151 (RDNA3.5):** 3.05x-3.20x gap (MAJOR)
- **gfx1100 (RDNA3):** 1.05x-1.13x gap (minor)
This is a genuine device-dependent codegen difference, not a version mismatch. Both devices use the **identical software stack**.
## Test Configuration
| Component | Version |
|-----------|---------|
| ROCm | TheRock 7.15.0a20260711 |
| AMD clang/LLVM | 23.0.0git, source `aa451e1f`, patch `440716f8` |
| Mesa/RADV | 26.1.4-arch3.1 |
| Kernel | linux-cachyos 7.1.3-2 |
| Firmware | linux-firmware 1:20260622-1 |
**Devices:**
- gfx1151: Radeon 8060S / Strix Halo (40 CUs)
- gfx1100: Radeon Pro W7900 (96 CUs)
**Workload parameters:**
- n=32768, body_iters=64, 16 packed groups/iter
- Workgroups: 64/256, samples: 10/3/5
**Wave configuration:**
- HIP: wave32
- Vulkan: wave64
- Programs are **operationally matched**, not textually identical
## Reproduction
Clone the benchmark suite:
```bash
git clone https://github.com/shisa-ai/hipEngine.git
cd hipEngine
# gfx1151 measured source
git checkout 0e566a4559b52a8bfc65ccdbda22556ae9112279
# Set up environment
bash scripts/update-therock-torch.sh \
--device gfx1151 --date 20260711 --test --verify-torch
export HIPENGINE_HIP_ARCH=gfx1151
export GPU_NAME="Radeon 8060S Graphics"
```
Run packed-integer benchmark (see [command templates](https://github.com/shisa-ai/hipEngine/blob/main/benchmarks/micro/README.md#current-gfx1151-command-templates)).
**Retained artifact:** [gfx1151 results JSON](https://github.com/shisa-ai/hipEngine/blob/f2c3ad6d74c86e3641ce09ff9fd759eaa6cd75e0/benchmarks/results/2026-07-11-gfx1151-hip-vulkan-matched-protocol.json)
## Requested Investigation
1. **Which AMD clang/LLVM team owns gfx11 packed-integer lowering and scheduling?**
2. **What artifacts are needed?**
- Preprocessed source
- LLVM IR/bitcode
- HSACO/disassembly
- Occupancy analysis
- Wait state analysis
- Issue counters
3. **Is there a preferred HIP builtin or source form for these loops?**
## Related Issues
**Parent issue:** [ROCm/ROCm#6409 - HIP/LLVM performance gaps vs RADV/Vulkan (3-part split)](https://github.com/ROCm/legacy-rocm-build/issues/6409)
- **Issue A (HIP Runtime):** Graph replay overhead (10x slower) https://github.com/ROCm/rocm-systems/issues/10834
- **Issue B (HIP Runtime):** Stream concurrency gap (21x slower on gfx1151) https://github.com/ROCm/rocm-systems/issues/10836
- **Issue C (LLVM - THIS ISSUE):** Packed-integer codegen performance
Contributor guide
Research direction
Start with the pinned hipEngine revision, the gfx1151 command templates in benchmarks/micro/README.md, and the retained results JSON. Reproduce the benchmark, then collect the requested preprocessed source, LLVM IR, HSACO/disassembly, occupancy, wait-state, and issue-counter artifacts. Done means identifying the AMDGPU codegen cause of the gap and documenting the relevant source form or investigation result.
Written by the indexing model from the issue text.
Assessment
- Domain
- compilers, performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100