deepseek-ai / deepseek-ai/DeepEP
Question about interface of low_latency_dispatch
- Dominant language
- Cuda
- Stars
- 10.1k
- Forks
- 1.4k
- Avg merge
- 4d 1h
- Merged PRs (30d)
- 2
Description
I'm a bit confused about the low_latency_dispatch interface. Why doesn't the interface **support passing topk_weights**? If the dispatch interface cannot return topk_weights, then it is necessary to
1. recalculate based on the obtained hidden stats
2. or use the dispatch interface again to transmit and obtain topk_weights.
However, based on our actual tests, the transmission time for all_gather during the decode stage is very short, so adding recalculations or additional transmissions does not offer any advantage. Or is it related to your experts' interface?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.