NVIDIA / NVIDIA/cutlass

[QST] Row Contiguous C Matrix from TiledMMA?

Open
#1,963 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

? - Needs Triage inactive-30d inactive-90d question
Dominant language
C++
Stars
10.5k
Forks
2.1k
Avg merge
3d 11h
Merged PRs (30d)
7

Description

What is your question?
Hello!

How do we get a row contiguous value layout from TiledMMA? That is, referring to this LaTeX value layout, we can see that values are interspersed across threads(I believe this helps coalescing?) How can we permute that value layout to achieve column-major contiguity, where Thread 0 owns Vals 0 to N-1; Thread 1, Vals N to 2N - 1 and so on?

My use case requires an independent epilogue operation performed on each row of the C matrix and, naturally, we can parallelize across threads. Currently, I permute the value layout by copying from registers to a shared memory workspace. However, this copy would be unnecessary, if the C matrix values were thread-row contiguous. In addition, I would also be able to operate on the register fragment directly rather than shared memory.

There probably is a memory access tradeoff lurking in the answer to this question, so please let me know if that is the case too :)

Thanks!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked CuTe LaTeX value-layout documentation and the TiledMMA layout involved in the C-matrix fragment. Compare its register ownership with the current copy through shared memory, then determine whether a row-contiguous layout or permutation is supported and document the resulting memory-access tradeoff.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.