deepseek-ai / deepseek-ai/DeepEP

Why the internode_ll::dispatch and internode_ll::combine use all SMs?

Open
#396 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Cuda
Stars
10.1k
Forks
1.4k
Avg merge
4d 1h
Merged PRs (30d)
2

Description

@LyricZhao , hello, I read the code for low latency:

https://github.com/deepseek-ai/DeepEP/blob/c18eabdebf1381978ff884d278f6083a6153be3f/csrc/deep_ep.cpp#L1169-L1182
https://github.com/deepseek-ai/DeepEP/blob/c18eabdebf1381978ff884d278f6083a6153be3f/csrc/deep_ep.cpp#L1274-L1285

I found that num_device_sms (the number of SMs) is from,
https://github.com/deepseek-ai/DeepEP/blob/c18eabdebf1381978ff884d278f6083a6153be3f/csrc/deep_ep.cpp#L43-L46

Does it means that all SMs are used for dispatch and combine? to minimize the time cost?

As all SMs are used, will this affect the compute-communication overlap? or we should assume that there is no computation as Experts are waiting for dispatch/combind?

Thanks.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.