Rollout observability: GRPO uploads 1-2 rollouts per step to W&B
Open
research
- Dominant language
- Python
- Stars
- 2k
- Forks
- 561
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 145
Description
Currently NeMo RL will [dump all the GRPO train rollouts to disk](https://github.com/NVIDIA-NeMo/RL/blob/main/nemo_rl/algorithms/grpo.py#L781) (thank you @terrykong for the pointer).
However, it would be great to have a more accessible way of getting rollout observability, namely in W&B. We should upload a small sample of rollouts to W&B (like the actual trajectories not just statistics). It can help catch sneaky and scary vllm issues like this one (found and shared by @ertkonuk) https://github.com/vllm-project/vllm/issues/17652
Contributor guide
Assessment
This issue has not been assessed yet.