DeepEP 17cf pin times out in multi-node normal dispatch with NVSHMEM 3.7.2
- Dominant language
- Python
- Stars
- 2k
- Forks
- 561
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 145
Description
## Summary
NeMo RL currently pins DeepEP to `17cfb817bccec3a9c247013360cc550c2bac441e` while allowing `nvidia-nvshmem-cu13>=3.6.5`. In the current nightly environment, this resolves to NVSHMEM 3.7.2.
With this combination, multi-node DeepEP normal dispatch consistently reaches its 100-second CPU timeout. The failure was first observed during the initial MoE policy logprob forward for Gemma 4 26B with EP16/CP2. The exact same error also occurs in two independent Qwen3.5 35B-A3B EP16 nightly tests, indicating a dependency-level multi-node DeepEP problem rather than a Gemma-specific issue.
Relevant dependency declarations on current `main`:
- [DeepEP pin](https://github.com/NVIDIA-NeMo/RL/blob/a366bc8cffec730080354415d74c2c12462ea028/pyproject.toml#L148)
- [NVSHMEM requirement](https://github.com/NVIDIA-NeMo/RL/blob/a366bc8cffec730080354415d74c2c12462ea028/pyproject.toml#L62)
## Observed error
The timeout occurs in the first inter-node dispatch:
```text
GroupedExpertsDeepEP.forward
-> token_permutation2
-> dispatch_all_to_all
-> fused_dispatch
-> deep_ep.Buffer.internode_dispatch
RuntimeError: DeepEP error: timeout (dispatch CPU)
```
Cleanup after the timeout can also report a CUDA illegal-memory-access error, but that appears downstream of the dispatch timeout.
## Nightly CI failures
[NVIDIA nightly pipeline 66217369](https://gitlab-master.nvidia.com/dl/JoC/nemo-ci/-/pipelines/66217369) contains two independent Qwen3.5 35B-A3B EP16 failures with the same signature:
- [LLM GRPO Qwen3.5 35B-A3B, 2 nodes, Automodel EP16 — job 425679893](https://gitlab-master.nvidia.com/dl/JoC/nemo-ci/-/jobs/425679893)
- [VLM GRPO Qwen3.5 35B-A3B Geo3K, 2 nodes, Automodel EP16 — job 425679909](https://gitlab-master.nvidia.com/dl/JoC/nemo-ci/-/jobs/425679909)
Both traces follow the same path through Automodel's `fused_a2a.py`, `buffer.dispatch`, and `Buffer.internode_dispatch` before raising `RuntimeError: DeepEP error: timeout (dispatch CPU)`.
## Controlled reproduction
We ran a minimal EP16 normal-dispatch and combine test using:
- Two H100 nodes, eight GPUs per node
- PyTorch 2.11.0+cu130
- CUDA 13.0
- NVSHMEM 3.7.2
- Gemma 4 26B routing dimensions:
- hidden size: 2816
- experts: 128
- top-k: 8
- tokens: 4096
The same container, environment, and test were used across separate H100 allocations.
| DeepEP implementation | Result |
|---|---|
| Current pin `17cfb81` | Failed after approximately 100 seconds with `timeout (dispatch CPU)` |
| Previous pin `29d31c0` containing DeepEP #564 | Passed; dispatch completed in 6.57 ms |
| Current pin `17cfb81` plus only the two #564 compatibility changes | Passed; dispatch completed in 9.60 ms |
The passing cases also completed the combine and correctness checks without CUDA, NCCL, assertion, or timeout errors.
NVSHMEM selected the `ibrc` transport and initialized IBGDA successfully in all three cases, so the failure does not appear to be an initial transport or GID configuration problem.
## Root cause
The result is experimentally isolated to the NVSHMEM compatibility changes from [DeepEP #564](https://github.com/deepseek-ai/DeepEP/pull/564):
1. Select the appropriate RC QP representation for the newer NVSHMEM internal layout.
2. Enable relocatable device code when compiling with NVSHMEM, preventing duplicate device-state definitions.
The current `17cfb81` hybrid-ep lineage does not contain these changes. Manually porting only those changes onto that revision makes the same test pass.
This is consistent with:
- [DeepEP #553](https://github.com/deepseek-ai/DeepEP/issues/553), where the same dispatch timeout was reported and #564 was confirmed as the fix.
- [DeepEP #707](https://github.com/deepseek-ai/DeepEP/issues/707), reporting legacy normal-mode dispatch timeouts with newer NVSHMEM.
- [DeepEP #696](https://github.com/deepseek-ai/DeepEP/pull/696), which also discusses incompatible NVSHMEM runtime layouts.
The regression entered NeMo RL when [PR #3436](https://github.com/NVIDIA-NeMo/RL/pull/3436) changed the x86 DeepEP pin from `29d31c0` to `17cfb81`.
## Proposed fix
- Port the #564-equivalent RC QP indexing and RDC fixes onto the pinned hybrid-ep revision, or update the pin to a hybrid-ep revision containing equivalent compatibility changes.
- Consider pinning NVSHMEM to an explicitly tested version or range instead of leaving the upper bound open.
- Add a two-node normal DeepEP dispatch smoke test to dependency or nightly validation. Single-node initialization does not expose this failure.
Contributor guide
Assessment
This issue has not been assessed yet.