deepseek-ai / deepseek-ai/DeepGEMM

[Question] Contiguous Grouped Gemm for CUDA Graph

Open
#235 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Cuda
Stars
7.8k
Forks
1.3k
Avg merge
3d 7h
Merged PRs (30d)
3

Description

Hi DeepSeek team,

I noticed that DeepEP provides an optional `num_worst_tokens` argument for CUDA Graph usage. Based on that, I assume contiguous Grouped GEMM is also intended to support CUDA Graph. However, it seems there is no input parameter to supply a valid m for the actual workload. While we can set `m_indices` to -1 for padded tokens, performance degrades significantly because `is_computation_valid` only skips the compute blocks, whereas the TMA loads are still issued for those padded regions.

Did I misunderstand something here, or is there a plan to address this limitation?

Thanks!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.