NVIDIA-NeMo / NVIDIA-NeMo/RL

TQ: GDR PUTs stage CPU-resident tensors through the GPU buffer

Open
#4,051 0 comments 0 reactions 1 assignee Claimed by @terrykong View on GitHub
bug Speed
Dominant language
Python
Stars
2k
Forks
561
Avg merge
4d 5h
Merged PRs (30d)
145

Description

Follow-up from #3501. Not introduced by it — the flag just exposes it.

TransferQueue selects the GDR path **per client, not per tensor**. Both PUT paths in NeMo-RL hand it CPU-resident tensors, so with `mooncake_cpu.use_gdr: true` those get staged through the GPU buffer — an extra H2D hop the CPU-RDMA path never makes — plus a `torch.cuda.synchronize()` per batch inside `_put_tensors_gdr` (`#L228`).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.