[BUG] cutlass-python does not work on H100 for CUDA 11.8
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 10.5k
- Forks
- 2.1k
- Avg merge
- 3d 11h
- Merged PRs (30d)
- 7
Description
Describe the bug
The following sample program fails with the latest version of cutlass:
import cutlass
import numpy as np
plan = cutlass.op.Gemm(element=np.float16, layout=cutlass.LayoutType.RowMajor)
A, B, C, D = [np.ones((1024, 1024), dtype=np.float16) for i in range(4)]
plan.run(A, B, C, D)
The error given is as follows:
Error Message:
nvcc fatal : Value 'sm_90a' is not defined for option 'gpu-architecture'
Steps/Code to reproduce bug
- Download the latest version of cutlass on a H100 node that is configured with CUDA 11.8.
- Build cutlass by following the installation instructions and (optionally) specify
-DCUTLASS_NVCC_ARCHS="90" - Install the python library by running
pip install .in the root directory of cutlass. - Run the example program.
Expected behavior
It looks like cutlass' python wrapper expects all H100 machines to want to run with CUDA 12+. I see, for instance, the following:
But that isn't necessarily the case.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with python/cutlass/emit/pytorch.py around lines 691-694 and reproduce the issue using the provided GEMM sample after building with CUDA 11.8 on an H100. Trace how the GPU architecture is selected, then verify that installing and running the Python example no longer produces the unsupported sm_90a error under CUDA 11.8.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- build-system, hpc
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100