deepseek-ai / deepseek-ai/DeepEP

Networking requirements(latency, bandwidth, throughput) for EP vs TP?

Open
#445 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Cuda
Stars
10.1k
Forks
1.4k
Avg merge
4d 1h
Merged PRs (30d)
2

Description

I'm curious about the differences in networking requirements between EP and TP, particularly regarding latency, bandwidth, and throughput. I know that both distribution strategies are highly demanding in terms of networking performance, but I'm not sure which one has higher requirements from latency, bandwidth, and throughput perspectives. Could someone elaborate on this a little for me?

I also know that NCCL wasn't well-optimized for all-to-all which EP requires, and that's why DeepEP was introduced. But why can NCCL meet TP's requirements? Is it because NCCL is well-suited for allreduce, or is it because TP is more tolerant of networking limitations compared to EP?

Thanks.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.