deepseek-ai / deepseek-ai/DeepEP
Why the internode_ll::dispatch and internode_ll::combine use all SMs?
- Dominant language
- Cuda
- Stars
- 10.1k
- Forks
- 1.4k
- Avg merge
- 4d 1h
- Merged PRs (30d)
- 2
Description
@LyricZhao , hello, I read the code for low latency:
https://github.com/deepseek-ai/DeepEP/blob/c18eabdebf1381978ff884d278f6083a6153be3f/csrc/deep_ep.cpp#L1169-L1182
https://github.com/deepseek-ai/DeepEP/blob/c18eabdebf1381978ff884d278f6083a6153be3f/csrc/deep_ep.cpp#L1274-L1285
I found that num_device_sms (the number of SMs) is from,
https://github.com/deepseek-ai/DeepEP/blob/c18eabdebf1381978ff884d278f6083a6153be3f/csrc/deep_ep.cpp#L43-L46
Does it means that all SMs are used for dispatch and combine? to minimize the time cost?
As all SMs are used, will this affect the compute-communication overlap? or we should assume that there is no computation as Experts are waiting for dispatch/combind?
Thanks.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.