[BUG] GRPO Nemotron HelpSteer3 recipe has very high logprob error
Open
bug
- Dominant language
- Python
- Stars
- 2k
- Forks
- 561
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 145
Description
**Describe the bug**
When running the nemotron-helpsteer3 GRPO recipe (examples/configs/recipes/llm/grpo-helpsteer3-llama-3.3-nemotron-super-49b-v1.5-4n8g-fsdp2tp8.yaml) with the HelpSteer3 dataset, the log_prob_error is extremely high both before and after PR [#1506](https://github.com/NVIDIA-NeMo/RL/pull/1506).
Contributor guide
Assessment
This issue has not been assessed yet.