deepseek-ai / deepseek-ai/DeepEP

[bug] kNumExpertsPerSM = 4 cause hung on H20

Open
#389 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Cuda
Stars
10.1k
Forks
1.4k
Avg merge
4d 1h
Merged PRs (30d)
2

Description

Hello,

Recently, we are trying to run deepep on some H20s and it always hung after some steps with recent commit (no such issue on older version, like before June 2nd). So we revert the commit one by one and reproduce this issue. Finally, we found it's https://github.com/deepseek-ai/DeepEP/commit/d4f3497 which introduced the issue:
`constexpr int kNumThreads = 256, kNumExpertsPerSM = 32, kNumRanksPerSM = 8;` => `constexpr int kNumThreads = 256, kNumExpertsPerSM = 4, kNumRanksPerSM = 8;`

Not sure why this happen. Anyone hit this issue?

Thanks,

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.