NVIDIA-NeMo / NVIDIA-NeMo/RL

[data-plane] cleanup tracking issue

Open
#2,509 0 comments 0 reactions 1 assignee Claimed by @ZhiyuLi-Nvidia View on GitHub
Dominant language
Python
Stars
2k
Forks
561
Avg merge
4d 5h
Merged PRs (30d)
145

Description

Tracking issue for loose ends in the data-plane / TQ integration (PR #2439 and follow-ups). Items will be checked off or split into their own issues as they are picked up.

### Known cleanup items

- [ ] `nemo_rl/algorithms/grpo_sync.py` — `grpo.calculate_advantages_on_gpu` is a no-op on the TQ path (TQ-resident tensors are CPU-side, baseline/std computed on CPU). Currently emits a `warnings.warn(...)` at runtime; decide whether to remove the flag or implement a GPU path.
- [ ] `nemo_rl/algorithms/grpo.py` — retire the legacy in-memory GRPO trainer once the sync TQ trainer is the default and async-on-TQ has landed.
- [ ] https://github.com/NVIDIA-NeMo/RL/pull/2439#discussion_r3262563156 (avoid padding and unpadding)
- [ ] ensure `claim_meta`, `get_data` and `check_consumption_status` is used in async loop (otherwise prune APIs)
- [ ] Add anything else here as it comes up.

Reference this issue from the `# TODO(#)` comments at the relevant sites.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.