deepseek-ai / deepseek-ai/DeepGEMM
m_grouped_gemm_fp8_fp8_bf16_nt_contiguous's requiresment of 128 padding for each expert causes computation waste
Open
- Dominant language
- Cuda
- Stars
- 7.8k
- Forks
- 1.3k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 3
Description
I noticed that the number of input tokens must be aligned to BLOCK_M (default 128) [code](https://github.com/deepseek-ai/DeepGEMM/blob/main/deep_gemm/include/deep_gemm/scheduler.cuh#L106), which can lead to up to 20% wasted computation when the workload is light. Would it be possible to relax this alignment requirement to 64?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.