deepseek-ai / deepseek-ai/DeepEP

Internode Combine Timeout

Open
#314 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Cuda
Stars
10.1k
Forks
1.4k
Avg merge
4d 1h
Merged PRs (30d)
2

Description

Hello! I found under this configuration, the **internode combine** will timeout: (The variable names I use here are the same as the ones used in `test_internode.py`)

**num_sms = 48
nvl_chunk_size = 32
nvl_buffer_size = 80
rdma_chunk_size = 20
rdma_buffer_size = 128**

For my experiment setup:
**Two nodes, each with 8 * H20 GPU and 4 * NIC, ROCE**

Could you try the config in your environment to confirm or deny this. Thanks a lot!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.