deepseek-ai / deepseek-ai/DeepGEMM
CUDA Illegal Instruction in test_k_grouped_gemm_contiguous
- Dominant language
- Cuda
- Stars
- 7.8k
- Forks
- 1.3k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 3
Description
I encountered a CUDA kernel error while running the `test_k_grouped_gemm_contiguous` test with the matrix dimensions explicitly set to:
* `M = 768`
* `N = 1536`
The test fails with the following error:
```text
torch.AcceleratorError: CUDA error: an illegal instruction was encountered
```
The error appears to originate from a CUDA kernel execution. Since CUDA errors can be reported asynchronously, the reported stack trace may not point to the actual source of the failure.
For debugging, I also considered using:
```bash
CUDA_LAUNCH_BLOCKING=1
```
and enabling device-side assertions with `TORCH_USE_CUDA_DSA`.
### Question
Is the `M=768, N=1536` shape combination currently supported by the grouped GEMM implementation?
If this shape is expected to be supported, could there be a kernel configuration or shape-related issue causing the illegal instruction?
### Environment
* Test: `test_k_grouped_gemm_contiguous`
* M: `768`
* N: `1536`
* Error: `CUDA error: an illegal instruction was encountered`
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the `test_k_grouped_gemm_contiguous` test using M=768 and N=1536, first reproducing it with `CUDA_LAUNCH_BLOCKING=1` as suggested. Determine whether the grouped GEMM implementation supports this shape and, if so, identify the shape or kernel configuration causing the illegal instruction; done means the support status or failure cause is established.
Written by the indexing model from the issue text.
Assessment
- Domain
- hpc
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100