NVIDIA / NVIDIA/cutlass

[FEA] Complete the cutlass::library::GemmDescription class to cover Hopper GEMM kernels

Open
#2,073 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

? - Needs Triage feature request inactive-30d inactive-90d
Dominant language
C++
Stars
10.5k
Forks
2.1k
Avg merge
3d 11h
Merged PRs (30d)
7

Description

Issue

CUTLASS GemmDevice Operator contains compile-time attributes (functional and performance attribute). The GemmDevice Operator is consumed by GemmOperation[3xBase]. In the past, I have found some of the values in this data structure incorrect and sometimes just completely missing.

For e.g. given the kernel by its full procedural name = cutlass3x_sm90_tensorop_s64x128x32gemm_e4m3_e4m3_f32_bf16_bf16_128x128x128_1x2x1_0_tnn_align16_warpspecialized_cooperative_epi_tma.

This kernel has the following functional and performance attribute:

Functional Attribute

  • dtypeA_dtypeB_dtypeAccumulation_dtypeC_dtypeD : e4m3_e4m3_f32_bf16_bf16
  • RowMajor_ColumnMajor_ColumnMajor : tnn

Performance Attribute

  • Instruction Shape : 64x128x32
  • Threadblock Shape : 128x128x128
  • Cluster Shape : 1x2x1
  • Mainloop Kind : warpspecialized_cooperative // This is missing from GemmDescription
  • Epilogue Kind : epi_tma // This is missing from GemmDescription
  • AccumulationKind : "fastaccum" or default "" // This is missing from GemmDescription

I have added a test so someone at NVIDIA can start on this. Can you please uncomment the two lines, add whatever is needed to fix this?

You can follow any other enum that is lifted up to GemmDevice Operator from internal templates and used to set the data members of GemmDescription class.

We can then commit this test and add more for Hopper and make sure this class is also covered for Blackwell. The tests are CPU-only and should not take too much time in the CI, this will allow us to catch bugs like this one.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with GemmDevice::Operator and GemmDescription in tools/library/src/gemm_operation_3x.hpp, then inspect the linked CPU-only test where the two assertions are commented out. Trace how existing enums are lifted from internal templates, add coverage for the Hopper attributes listed in the issue, and verify the test passes with the expected metadata.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend-api-design, performance, testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.