mma_scaled accumulated over a K loop faults with cudaErrorMisalignedAddress on sm_103

Three conditions are needed as none of them alone reproduces it.

• A scale tile whose K extent is 8, i.e. block_k / B == 8.
• An even number of loop iterations. 1 and 3 are clean and return the correct value; 2 and 4 fault.
• An output tile wider than 16. block_n=16 is clean, 64 and 128 fault, and block_n=32 fails differently, with TileCompilerExecutionError at compile time rather than a fault at synchronize.

import subprocess
import sys

import cupy as cp
import cuda.tile as ct

FORMATS = {
    "nvfp4": (True, ct.float8_e4m3fn, 16),
    "mxfp4": (True, ct.float8_e8m0fnu, 32),
    "mxfp8": (False, ct.float8_e8m0fnu, 32),
}

if len(sys.argv) == 1:
    cases = [(n, e * FORMATS[n][2], 2, 128) for n in FORMATS for e in (4, 8, 16)]
    cases += [("mxfp8", 256, i, 128) for i in (1, 3, 4)]
    cases += [("mxfp8", 256, 2, bn) for bn in (16, 32, 64)]
    for case in cases:
        done = subprocess.run([sys.executable, __file__, *map(str, case)],
                              capture_output=True, text=True)
        print(done.stdout.strip())
    sys.exit(0)

NAME, BK, ITERS, BN = sys.argv[1], int(sys.argv[2]), int(sys.argv[3]), int(sys.argv[4])
FP4, SCALE, B = FORMATS[NAME]
K = ITERS * BK


@ct.kernel
def gemm(a, a_s, b, b_s, out):
    acc = ct.zeros((128, BN), dtype=ct.float32)
    for k in range(ITERS):
        if FP4:
            av = ct.reshape(ct.unpack_from_bytes(
                ct.reshape(ct.load(a, (0, k), (128, BK // 2)), (128 * BK // 2,)),
                ct.float4_e2m1fn), (128, BK))
            bv = ct.reshape(ct.unpack_from_bytes(
                ct.reshape(ct.load(b, (k, 0), (BK, BN // 2)), (BK * BN // 2,)),
                ct.float4_e2m1fn), (BK, BN))
        else:
            av = ct.bitcast(ct.load(a, (0, k), (128, BK)), ct.float8_e4m3fn)
            bv = ct.bitcast(ct.load(b, (k, 0), (BK, BN)), ct.float8_e4m3fn)
        acc = ct.mma_scaled(
            av, ct.bitcast(ct.load(a_s, (0, k), (128, BK // B)), SCALE),
            bv, ct.bitcast(ct.load(b_s, (k, 0), (BK // B, BN)), SCALE),
            acc)
    ct.store(out, (0, 0), acc)


u8 = lambda *s: cp.zeros(s, dtype=cp.uint8)
args = (u8(128, K // 2 if FP4 else K), u8(128, K // B),
        u8(K, BN // 2 if FP4 else BN), u8(K // B, BN),
        cp.zeros((128, BN), dtype=cp.float32))
info = (f"{NAME}  {'f4e2m1fn' if FP4 else 'f8e4m3fn'}  B={B:<3} block_k={BK:<4} "
        f"scale K extent={BK // B:<3} iters={ITERS:<2} block_n={BN:<4}")

try:
    ct.launch(cp.cuda.get_current_stream().ptr, (1, 1), gemm, args)
    cp.cuda.runtime.deviceSynchronize()
except Exception as exc:
    print(f"{info}  {'FAULT' if 'isaligned' in str(exc) else type(exc).__name__}")
    sys.exit(1)
print(f"{info}  ok")

One case per process, because the fault is sticky:

nvfp4  f4e2m1fn  B=16  block_k=64   scale K extent=4   iters=2  block_n=128   ok
nvfp4  f4e2m1fn  B=16  block_k=128  scale K extent=8   iters=2  block_n=128   FAULT
nvfp4  f4e2m1fn  B=16  block_k=256  scale K extent=16  iters=2  block_n=128   ok
mxfp4  f4e2m1fn  B=32  block_k=128  scale K extent=4   iters=2  block_n=128   ok
mxfp4  f4e2m1fn  B=32  block_k=256  scale K extent=8   iters=2  block_n=128   FAULT
mxfp4  f4e2m1fn  B=32  block_k=512  scale K extent=16  iters=2  block_n=128   ok
mxfp8  f8e4m3fn  B=32  block_k=128  scale K extent=4   iters=2  block_n=128   ok
mxfp8  f8e4m3fn  B=32  block_k=256  scale K extent=8   iters=2  block_n=128   FAULT
mxfp8  f8e4m3fn  B=32  block_k=512  scale K extent=16  iters=2  block_n=128   ok
mxfp8  f8e4m3fn  B=32  block_k=256  scale K extent=8   iters=1  block_n=128   ok
mxfp8  f8e4m3fn  B=32  block_k=256  scale K extent=8   iters=3  block_n=128   ok
mxfp8  f8e4m3fn  B=32  block_k=256  scale K extent=8   iters=4  block_n=128   FAULT
mxfp8  f8e4m3fn  B=32  block_k=256  scale K extent=8   iters=2  block_n=16    ok
mxfp8  f8e4m3fn  B=32  block_k=256  scale K extent=8   iters=2  block_n=32    TileCompilerExecutionError
mxfp8  f8e4m3fn  B=32  block_k=256  scale K extent=8   iters=2  block_n=64    FAULT

A plain ct.mma in the same loop is correct at every tile width and iteration count, which points at the scale operand path rather than the mma itself.

cuda-tile 1.5.0, nvidia-cuda-tileiras 13.3.36, NVIDIA GB300 (sm_103).

mma_scaled accumulated in a loop faults at deviceSynchronize with cudaErrorMisalignedAddress. Inputs are all zeros, so this is addressing rather than data.

Start by running the inline Python reproducer one case per process on the stated GB300 environment, focusing on the ct.mma_scaled loop and comparing it with the plain ct.mma case. Trace the scale-operand path for scale K extent 8, even iteration counts, and output widths above 16; done means the listed cases no longer fault or fail compilation and still produce correct results.