[BUG] SM90 blockwise FP8 GEMM reuses modified B scales across M waves
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 10.5k
- Forks
- 2.1k
- Avg merge
- 3d 11h
- Merged PRs (30d)
- 7
Description
Which component has the problem?
CUTLASS C++
Bug Report
Describe the bug
SM90 cooperative blockwise FP8 GEMM applies the preceding M wave's A scale again when processing the next M wave. With CTA tile 256×128×128 and scale granularity 128×64×128, D[128,0] is 1024 instead of 512.
Steps/Code to reproduce bug
In a checkout of 59e3a3338d516ca6ce0e073af8da65289678a35c:
curl -fL https://raw.githubusercontent.com/1sgtpepper/cutlass/75e7b4a8a2859c317209d81c91c5ab9158a573e7/test/regression/fp8_scale_wave.cu -o fp8_scale_wave.cu
nvcc -std=c++17 -O3 -lineinfo --expt-relaxed-constexpr \
-gencode=arch=compute_90a,code=sm_90a \
-Iinclude -Itools/util/include fp8_scale_wave.cu -o fp8_scale_wave
./fp8_scale_wave
Inputs are FP8 E4M3 ones, with A scales 2 and 4 across the row bands and B scales 1 and 2 across the column bands.
minimal l=0 m=128 n=0 actual=1024 expected=512
minimal tile=256x128x128 scale=128x64x128 stages=3 problem=256x128x128x1 mismatches=16384 FAIL
Expected behavior
Each M wave multiplies its own A scale by the original B scale for the current K tile.
Environment details (please complete the following information):
- CUTLASS
59e3a3338d516ca6ce0e073af8da65289678a35c, main as of September 8, 2026. The affected collective is byte-identical inv4.8.0devandv4.7.1; these releases were not executed. - H100 80GB HBM3, driver 580.95.05; CUDA 13.1.1.
- Modal cloud GPU, Ubuntu 24.04 container, Linux x86_64.
Additional context
The ScaleMsPerWave == 1 && ScaleNsPerTile > 1 path modifies B-scale registers reused by successive M waves, in both the main loop and drain. Reloading B's original scales makes all ten --all cases pass; restoring the old code restores the same six failures.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Download test/regression/fp8_scale_wave.cu from the linked revision and compile it with the provided nvcc command on an SM90 GPU. Then trace the SM90 cooperative blockwise FP8 GEMM collective through its main loop and drain, focusing on the ScaleMsPerWave == 1 and ScaleNsPerTile > 1 path. Done means the reproduction no longer reports mismatches and all ten --all cases pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100