NVIDIA-NeMo / NVIDIA-NeMo/RL

Lack of education on good use case of RL, how we should go use the examples, and how we customize it if needed

Open
#1,504 0 comments 0 reactions 0 assignees View on GitHub
Documentation i-tme
Dominant language
Python
Stars
2k
Forks
561
Avg merge
4d 5h
Merged PRs (30d)
145

Description

**Is your feature request related to a problem? Please describe.**
There is a lack of clear education and onboarding material around how and when to use Reinforcement Learning in NeMo RL. The [main documentation page](https://docs.nvidia.com/nemo/rl/latest/index.html)
does not provide an overview of which types of problems RL is best suited for (e.g., Classification and structured prediction, domain adaptation, or multi-step processes, etc.), nor does it help users decide when RL is appropriate versus other training methods(e.g., SFT).

Additionally, the example workflows (such as the math example) lack introductory guidance on how to adapt them for new use cases. Understanding which parts of the example need modification—environment.py, run_grpo_math.py, reward functions, or configuration files—is not straightforward.

There is also ambiguity around how to run YAML configurations. For instance, details about enabling checkpoint, validation period, system prompt file path are buried deep in the “Design Doc” section, making it difficult to find essential information.

**Describe the solution you'd like**
1. Adding a “When to Use RL” section on the main NeMo RL page.
2. Including an “Customizing This Example” section for each tutorial so that users could locate and change reward function, chat template, etc.
3. Centralizing YAML usage including checkpoint enabling details in a visible, beginner-friendly section.

**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.

**Additional context**
Pages I looked at: https://docs.nvidia.com/nemo/rl/latest/index.html, https://docs.nvidia.com/nemo/rl/latest/guides/grpo-deepscaler.html, https://docs.nvidia.com/nemo/rl/latest/guides/grpo.html

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.