deepseek-ai / deepseek-ai/DeepGEMM
v2 seems to be slower than v1 on hopper on some cases
- Dominant language
- Cuda
- Stars
- 7.8k
- Forks
- 1.3k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 3
Description
related: https://github.com/sgl-project/sglang/pull/9167
I briefly look at profile, and see some deepgemm kernels have slower speed. before digging deeper, I create an issue to report to know e.g. is it on the roadmap and is not yet optimized.
speed:
baseline:
[1755163919.4467359-TP-6.trace.json.gz](https://github.com/user-attachments/files/21770886/1755163919.4467359-TP-6.trace.json.gz)
pr:
[1755161669.4129217-TP-6.trace.json.gz](https://github.com/user-attachments/files/21770892/1755161669.4129217-TP-6.trace.json.gz)
example screenshot (for a stable-shape gemm. the most time-consuming gemm is moe, but that changes a lot since expert is imbalanced, so I do not screenshot here)
baseline:
pr:
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.