deepseek-ai / deepseek-ai/DeepEP

Question about interface of low_latency_dispatch

Open
#72 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Cuda
Stars
10.1k
Forks
1.4k
Avg merge
4d 1h
Merged PRs (30d)
2

Description

I'm a bit confused about the low_latency_dispatch interface. Why doesn't the interface **support passing topk_weights**? If the dispatch interface cannot return topk_weights, then it is necessary to
1. recalculate based on the obtained hidden stats
2. or use the dispatch interface again to transmit and obtain topk_weights.
However, based on our actual tests, the transmission time for all_gather during the decode stage is very short, so adding recalculations or additional transmissions does not offer any advantage. Or is it related to your experts' interface?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.