NVIDIA / NVIDIA/cutlass

[BUG] domain_offset overflows runtime Int32 coordinate with static stride

Open
#3,604 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

CuTe DSL
Dominant language
C++
Stars
10.5k
Forks
2.1k
Avg merge
3d 11h
Merged PRs (30d)
7

Description

Which component has the problem?

CuTe DSL

Bug Report

Describe the bug

cute.domain_offset can overflow a runtime Int32 coordinate when the tensor has a static stride. The element offset is calculated in 32 bits before conversion to a 64-bit byte offset, which can cause CUDA_ERROR_ILLEGAL_ADDRESS. The equivalent dynamic-stride layout uses 64-bit arithmetic and succeeds.

Steps/Code to reproduce bug

import argparse
import os

os.environ.setdefault("CUDA_LAUNCH_BLOCKING", "1")

import torch
import cutlass
import cutlass.cute as cute
from cutlass.cute.runtime import from_dlpack


ROWS = 2**29 + 1
COLS = 4
LAST_ROW = ROWS - 1


# IMA: from_dlpack(out) preserves the fully static compact layout:
#   !cute.layout<"(536870913,4):(4,1)">
@cute.kernel
def no_make_layout_static_stride(tensor: cute.Tensor, row: cutlass.Int32):
    # IR emitted by domain_offset:
    #   %idx = cute.crd2idx(%coord, %lay)
    #       : (...) -> !cute.int_tuple<"?{div=4}">
    #   %ptr = cute.add_offset(%iter, %int_tuple)
    shifted = cute.domain_offset((row, 0), tensor)
    shifted[(0, 0)] = cutlass.BFloat16(1.0)


# Works: the runtime row stride remains i64.
@cute.kernel
def make_layout_dynamic_stride(tensor: cute.Tensor, row: cutlass.Int32):
    layout = cute.make_layout(
        (tensor.shape[0], COLS),
        stride=(tensor.stride[0], 1),
    )
    # IR for layout:
    #   %lay_3 = cute.make_layout(%shape, %stride)
    #       : !cute.layout<"(?,4):(?{i64},1)">
    rebuilt = cute.make_tensor(tensor.iterator, layout)
    # IR emitted by domain_offset:
    #   %idx = cute.crd2idx(%coord, %lay_3)
    #       : (...) -> !cute.int_tuple<"?{i64}">
    #   %ptr = cute.add_offset(%iter, %int_tuple)
    shifted = cute.domain_offset((row, 0), rebuilt)
    shifted[(0, 0)] = cutlass.BFloat16(1.0)


# IMA: make_layout replaces the dynamic stride with the static stride (4, 1).
@cute.kernel
def make_layout_static_stride(tensor: cute.Tensor, row: cutlass.Int32):
    layout = cute.make_layout(
        (tensor.shape[0], COLS),
        stride=(COLS, 1),
    )
    # IR for layout:
    #   %lay_0 = cute.make_layout(%shape, %stride)
    #       : !cute.layout<"(?,4):(4,1)">
    rebuilt = cute.make_tensor(tensor.iterator, layout)
    # IR emitted by domain_offset:
    #   %idx = cute.crd2idx(%coord, %lay_0)
    #       : (...) -> !cute.int_tuple<"?{div=4}">
    #   %ptr = cute.add_offset(%iter, %int_tuple)
    shifted = cute.domain_offset((row, 0), rebuilt)
    shifted[(0, 0)] = cutlass.BFloat16(1.0)


@cute.jit
def launch_no_make_layout_static_stride(tensor: cute.Tensor, row: cutlass.Int32):
    no_make_layout_static_stride(tensor, row).launch(grid=[1, 1, 1], block=[1, 1, 1])


@cute.jit
def launch_make_layout_dynamic_stride(tensor: cute.Tensor, row: cutlass.Int32):
    make_layout_dynamic_stride(tensor, row).launch(grid=[1, 1, 1], block=[1, 1, 1])


@cute.jit
def launch_make_layout_static_stride(tensor: cute.Tensor, row: cutlass.Int32):
    make_layout_static_stride(tensor, row).launch(grid=[1, 1, 1], block=[1, 1, 1])


cases = (
    "no_make_layout_static_stride",
    "make_layout_dynamic_stride",
    "make_layout_static_stride",
)
parser = argparse.ArgumentParser()
parser.add_argument("case", choices=cases)
args = parser.parse_args()

out = torch.zeros((ROWS, COLS), device="cuda", dtype=torch.bfloat16)
row = cutlass.Int32(LAST_ROW)

if args.case == "no_make_layout_static_stride":
    launch_no_make_layout_static_stride(from_dlpack(out), row)
elif args.case == "make_layout_dynamic_stride":
    launch_make_layout_dynamic_stride(from_dlpack(out).mark_layout_dynamic(), row)
else:
    launch_make_layout_static_stride(from_dlpack(out).mark_layout_dynamic(), row)

torch.cuda.synchronize()
assert float(out[0, 0]) == 0.0
assert float(out[-1, 0]) == 1.0
$ python repro.py no_make_layout_static_stride
CUDA_ERROR_ILLEGAL_ADDRESS (error code: 700)

$ python repro.py make_layout_dynamic_stride
# succeeds

$ python repro.py make_layout_static_stride
CUDA_ERROR_ILLEGAL_ADDRESS (error code: 700)

Expected behavior

All three cases should write out[-1, 0] without an illegal memory access.

Environment details

  • Docker
  • nvidia-cutlass-dsl==4.8.0a0+20260828210702.38664cd
  • CUDA 13.4
  • GB300 (sm_103a)

Additional context

The static-stride path lowers the address calculation as:

shl.b32      %r2, %r1, 2;
mul.wide.s32 %rd2, %r2, 2;
add.s64      %rd3, %rd1, %rd2;

For row = 2**29, the first instruction produces 0x80000000; signed widening then generates a -4 GiB byte offset. BF16 is used here only to expose the incorrect index-width propagation through element-to-byte lowering; the coordinate-to-index issue is not BF16-specific. The dynamic-stride case keeps the index in 64 bits:

mul.lo.s64 %rd4, %rd2, %rd3;
shl.b64    %rd5, %rd4, 1;
add.s64    %rd6, %rd1, %rd5;

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the supplied repro.py cases and inspect the lowering path for cute.domain_offset when a layout has a static stride, comparing it with the dynamic-stride path and the emitted PTX. Done means all three cases write out[-1, 0] successfully without CUDA_ERROR_ILLEGAL_ADDRESS and preserve the existing assertions.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
backend, compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.