TQ: GDR PUTs stage CPU-resident tensors through the GPU buffer
Open
bug
Speed
- Dominant language
- Python
- Stars
- 2k
- Forks
- 561
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 145
Description
Follow-up from #3501. Not introduced by it — the flag just exposes it.
TransferQueue selects the GDR path **per client, not per tensor**. Both PUT paths in NeMo-RL hand it CPU-resident tensors, so with `mooncake_cpu.use_gdr: true` those get staged through the GPU buffer — an extra H2D hop the CPU-RDMA path never makes — plus a `torch.cuda.synchronize()` per batch inside `_put_tensors_gdr` (`#L228`).
Contributor guide
Assessment
This issue has not been assessed yet.