NVIDIA / NVIDIA/cutlass

[BUG] SM90 blockwise FP8 GEMM reuses modified B scales across M waves

Open
#3,596 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
10.5k
Forks
2.1k
Avg merge
3d 11h
Merged PRs (30d)
7

Description

Which component has the problem?

CUTLASS C++

Bug Report

Describe the bug

SM90 cooperative blockwise FP8 GEMM applies the preceding M wave's A scale again when processing the next M wave. With CTA tile 256×128×128 and scale granularity 128×64×128, D[128,0] is 1024 instead of 512.

Steps/Code to reproduce bug

In a checkout of 59e3a3338d516ca6ce0e073af8da65289678a35c:

curl -fL https://raw.githubusercontent.com/1sgtpepper/cutlass/75e7b4a8a2859c317209d81c91c5ab9158a573e7/test/regression/fp8_scale_wave.cu -o fp8_scale_wave.cu
nvcc -std=c++17 -O3 -lineinfo --expt-relaxed-constexpr \
  -gencode=arch=compute_90a,code=sm_90a \
  -Iinclude -Itools/util/include fp8_scale_wave.cu -o fp8_scale_wave
./fp8_scale_wave

Inputs are FP8 E4M3 ones, with A scales 2 and 4 across the row bands and B scales 1 and 2 across the column bands.

minimal l=0 m=128 n=0 actual=1024 expected=512
minimal tile=256x128x128 scale=128x64x128 stages=3 problem=256x128x128x1 mismatches=16384 FAIL

Expected behavior

Each M wave multiplies its own A scale by the original B scale for the current K tile.

Environment details (please complete the following information):

  • CUTLASS 59e3a3338d516ca6ce0e073af8da65289678a35c, main as of September 8, 2026. The affected collective is byte-identical in v4.8.0dev and v4.7.1; these releases were not executed.
  • H100 80GB HBM3, driver 580.95.05; CUDA 13.1.1.
  • Modal cloud GPU, Ubuntu 24.04 container, Linux x86_64.

Additional context

The ScaleMsPerWave == 1 && ScaleNsPerTile > 1 path modifies B-scale registers reused by successive M waves, in both the main loop and drain. Reloading B's original scales makes all ten --all cases pass; restoring the old code restores the same six failures.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Download test/regression/fp8_scale_wave.cu from the linked revision and compile it with the provided nvcc command on an SM90 GPU. Then trace the SM90 cooperative blockwise FP8 GEMM collective through its main loop and drain, focusing on the ScaleMsPerWave == 1 and ScaleNsPerTile > 1 path. Done means the reproduction no longer reports mismatches and all ten --all cases pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.