NVIDIA / NVIDIA/cutlass

[BUG]

Open
#1,967 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

? - Needs Triage bug inactive-30d inactive-90d
Dominant language
C++
Stars
10.5k
Forks
2.1k
Avg merge
3d 11h
Merged PRs (30d)
7

Description

Describe the bug
split_k_slices option does not appear to do anything on the profiler.

Steps/Code to reproduce bug

./cutlass_profiler --operation=Gemm --m=256 --n=256 --k=8192 --A=f16:column --B=f16:row --C=f16:column --split_k_slices=1,2,4,8,16

Expected behavior
For such a skinny GEMM, split_k should have a significant effect on the performance. But there is not difference in performance between the different splits.

Environment details (please complete the following information):

  • Bare-metal, H100 PCIe
  • CUTLASS 3.6.0

Additional context
Both at 1 and 16 splits the profiler measured ~26900 GFLOP/s on our device. The best performing ones for the two splits were:

# 1 split
1,CUTLASS,gemm,cutlass3x_sm90_sptensorop_s64x128x32spgemm_f16_f16_f32_void_f16_128x128x64_1x2x1_0_ntn_align8_warpspecialized_cooperative_epi_nosmem,passed,success,universal,256,256,8192,f16:column,f16:row,f16:column,f16:column,1,0,serial,1,1,heuristic,1,sptensorop,f32,128,128,64,1,2,1,9,4,2,1,64,128,32,90,90,8519680,1073872896,126,0.0398454,199.134,26951
# 16 splits
5,CUTLASS,gemm,cutlass3x_sm90_sptensorop_s64x128x32spgemm_f16_f16_f32_void_f16_128x128x64_1x2x1_0_ntn_align8_warpspecialized_cooperative_epi_nosmem,passed,success,universal,256,256,8192,f16:column,f16:row,f16:column,f16:column,1,0,serial,16,1,heuristic,1,sptensorop,f32,128,128,64,1,2,1,9,4,2,1,64,128,32,90,90,8519680,1073872896,126,0.039881,198.956,26927

We also tried less extreme shapes, but they all seemed to disregards split_k_slices.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the report with the provided cutlass_profiler GEMM command on the stated H100 and compare results for split_k_slices values from 1 through 16. Trace how the profiler passes and reports split_k_slices, then verify that the selected configurations or measured performance reflect the requested splits rather than remaining identical.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
performance, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.