deepseek-ai / deepseek-ai/DeepEP
Can deep_ep run on environments with more than 160 ranks?
- Dominant language
- Cuda
- Stars
- 10.1k
- Forks
- 1.4k
- Avg merge
- 4d 1h
- Merged PRs (30d)
- 2
Description
I noticed a restriction in /csrc/deep_ep.cpp:
EP_HOST_ASSERT(0 <= rank && rank < num_ranks &&
(num_ranks <= NUM_MAX_NVL_PEERS * NUM_MAX_RDMA_PEERS || low_latency_mode));
where NUM_MAX_NVL_PEERS = 8 and NUM_MAX_RDMA_PEERS = 20. This implies the rank count cannot exceed 160 (8*20).
I tested this on a 24-node cluster, and the assertion was triggered. Therefore, my question is:
Does deep_ep actually support training on clusters with more than 20 nodes? Or is there any misunderstanding in my interpretation?
Contributor guide
No contributing guide indexed for this repository
Research direction
Read csrc/deep_ep.cpp, focusing on EP_HOST_ASSERT and the NUM_MAX_NVL_PEERS and NUM_MAX_RDMA_PEERS limits. Reproduce the 24-node configuration or trace how rank counts are validated; done means establishing whether ranks above 160 are supported and documenting the supported behavior or agreed direction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100