NVIDIA / NVIDIA/cutlass

add_tile_offset cross-term is not scaled by the element size in most transform/threadblock access iterators

Open
#3,487 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

CUTLASS C++
Dominant language
C++
Stars
10.5k
Forks
2.1k
Avg merge
3d 11h
Merged PRs (30d)
7

Description

Which component has the problem?

CUTLASS C++ - include/cutlass/transform/threadblock/predicated_tile_access_iterator.h and siblings

Describe the bug

add_tile_offset(TensorCoord const &tile_offset) moves the iterator by whole tiles along both ranks. When a caller passes a non-zero contiguous() component while advancing along the strided rank (or vice versa), that cross-term must be scaled by the element size because the iterators keep byte-based pointers. The residue-handling branch of the plain predicated tile access iterator does scale it, but every other path applies the raw tile count:

include/cutlass/transform/threadblock/predicated_tile_access_iterator.h, residue path (correct):

pointer_ += Shape::kContiguous * tile_offset.contiguous() * sizeof_bits<Element>::value / 8;
...
pointer_ += Shape::kStrided * tile_offset.strided() * sizeof_bits<Element>::value / 8;

same function, steady-state path a few lines below (unscaled):

pointer_ += Shape::kContiguous * tile_offset.contiguous();
...
pointer_ += Shape::kStrided * tile_offset.strided();

So the same call arguments take differently-scaled code paths depending on is_residual_tile_. The same unscaled pattern appears in:

  • predicated_tile_access_iterator_2dthreadtile.h (both paths)
  • ell_predicated_tile_access_iterator.h (both paths)
  • predicated_tile_access_iterator_triangular_matrix.h (both paths)
  • predicated_tile_access_iterator.h, AffineRankN<2> specialization, which additionally ignores stride_[0] for the contiguous rank

I believe no in-tree caller currently trips this: every add_tile_offset call site on these global-memory iterators passes exactly one non-zero component ({1,0} / {0,1} / {0,-1} style), so the cross term is always zero and the missing scaling is invisible. It is nevertheless a broken API contract for any external code moving tiles diagonally with Element wider than one byte, and the intra-file inconsistency shows at least one of the two paths must be wrong.

Steps to reproduce

Static: diff the two branches of PredicatedTileAccessIterator::add_tile_offset in the file above. A host-side demonstration needs only instantiating the iterator with a two-component tile offset, which no current example does.

Expected behavior

Both paths should apply the same, correctly scaled cross-term (element size, and for the AffineRankN<2> case the layout stride).

Environment details
  • Current main, header-only observation
  • Found while auditing the transform/threadblock iterator family; no functional failure observed in any in-tree kernel

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing add_tile_offset in include/cutlass/transform/threadblock/predicated_tile_access_iterator.h and its residue and steady-state branches. Then inspect predicated_tile_access_iterator_2dthreadtile.h, ell_predicated_tile_access_iterator.h, predicated_tile_access_iterator_triangular_matrix.h, and the AffineRankN<2> specialization. Done means all identified paths handle diagonal offsets with element-size scaling and the specialization respects layout stride; a host-side two-component instantiation can validate the behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
hpc
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.