NVIDIA-NeMo/RL

View on GitHub

Scalable toolkit for efficient model reinforcement

Stars: 2k
Forks: 562
Open beginner issues: 0
Indexed issues: 529
Avg merge: 4d 5h
Merged PRs (30d): 145

Dominant language: Python
License: Apache-2.0
Last GitHub push: Sep 18, 2026
Latest indexed: Sep 17, 2026
Contributing guide: Contributing guide
Code of conduct: No code of conduct
Beginner labels: good first issue

Open All

529 open issues indexed

torch.compile for training Open

Performance t-pytdensor

NVIDIA-NeMo/RL#4 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Tool calling Open

environments

NVIDIA-NeMo/RL#55 · 12 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Mixtral support (8x7b) in Dtensor path Open

new model t-pytdensor

NVIDIA-NeMo/RL#63 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Environment post-processing (reward scaling and metrics) Open

environments evaluation

NVIDIA-NeMo/RL#65 · 2 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Improve refitting (avoid all-gather) Open

inference Performance vllm

NVIDIA-NeMo/RL#70 · 1 comment · 0 reactions · 3 assignees · Indexed Sep 17, 2026
Avoid vLLM copy (doubles the memory) Open

inference Performance vllm

NVIDIA-NeMo/RL#72 · 1 comment · 0 reactions · 3 assignees · Indexed Sep 17, 2026
[General] Improve Logging Open

UX

NVIDIA-NeMo/RL#73 · 3 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Dependency Isolation (containers) Open

Infra

NVIDIA-NeMo/RL#103 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Allow freezing/unfreezing config Open

UX

NVIDIA-NeMo/RL#106 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Investigate apply_chat_template per message vs. all messages Open

NVIDIA-NeMo/RL#107 · 1 comment · 1 reaction · 0 assignees · Indexed Sep 17, 2026
`use_reference_model` is much slower than expected Open

bug Performance t-pytdensor

NVIDIA-NeMo/RL#144 · 8 comments · 1 reaction · 1 assignee · Indexed Sep 17, 2026
dtensor foreach must be false in optimizer Open

bug t-pytdensor

NVIDIA-NeMo/RL#151 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Avoid TP gather in streaming Open

vllm

NVIDIA-NeMo/RL#164 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Enabling cuda graph leads to convergence issue Open

bug vllm

NVIDIA-NeMo/RL#186 · 2 comments · 0 reactions · 2 assignees · Indexed Sep 17, 2026
Support additional preference-based objectives Open

external x-tii

NVIDIA-NeMo/RL#193 · 3 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Support LLama 4 in Megatron Backend Open

new model t-mcore

NVIDIA-NeMo/RL#201 · 2 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Code Coverage Reporting Open

NVIDIA-NeMo/RL#220 · 2 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
High vllm memory after sleep with tp > 1 Open

bug vllm

NVIDIA-NeMo/RL#223 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Enable liger kernels in DTensorPolicyWorkerV2 Open

t-pytdensor

NVIDIA-NeMo/RL#224 · 7 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
vllm seed resume Open

bug vllm

NVIDIA-NeMo/RL#226 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Refactor and clean up codebase Open

UX

NVIDIA-NeMo/RL#228 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
cumemallocator Open

vllm

NVIDIA-NeMo/RL#286 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
FSDP Worker (single-GPU) optimizations Open

Performance t-pytdensor

NVIDIA-NeMo/RL#313 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
token batch size rather than GBS Open

NVIDIA-NeMo/RL#332 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Additional Algorithms Support Open

algorithm external x-google

NVIDIA-NeMo/RL#337 · 3 comments · 1 reaction · 0 assignees · Indexed Sep 17, 2026
[Feature Request] Add configurable `resume_if_exists` flag to control automatic checkpoint resumption Open

enhancement external t-checkpoint x-abeja

NVIDIA-NeMo/RL#345 · 4 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Create RL playbook as a follow up to DAPT Open

enhancement good first issue

NVIDIA-NeMo/RL#404 · 5 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
[Feature] Explicit failure for unmatched model and checkpoints Open

bug

NVIDIA-NeMo/RL#415 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
gemma-3-4b-it got nan probs_ratio in both FSDP1/FSDP2 Open

bug t-pytdensor

NVIDIA-NeMo/RL#419 · 5 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Notebook series Open

NVIDIA-NeMo/RL#428 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Sglang support Open

NVIDIA-NeMo/RL#429 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Feature to mark invalid outputs from the reward system Open

NVIDIA-NeMo/RL#431 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Support `exp_manager.max_time_per_run` to guarantee save before deadline Open

NVIDIA-NeMo/RL#444 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Use async vllm for multi-turn and reward pipelining Open

enhancement vllm

NVIDIA-NeMo/RL#447 · 3 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Create a separate `eval` function to replace `train` + `eval_mode=True` Open

NVIDIA-NeMo/RL#448 · 2 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Add bitsandbytes to dependencies Open

NVIDIA-NeMo/RL#471 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Investigate cuda base version skew with stable torch versions Open

NVIDIA-NeMo/RL#486 · 2 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Add safe_serialization (safetensors) option to the checkpoint conversion script (`examples/convert_dcp_to_hf.py`) Open

enhancement

NVIDIA-NeMo/RL#491 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
NeMo Run Guides Open

external Stale x-Lila x-paypal x-zm

NVIDIA-NeMo/RL#531 · 3 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Evaluation results differ when `num_prompts_per_step` is different Open

bug external x-google

NVIDIA-NeMo/RL#563 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Does this project require CUDA 12.8? Open

external question x-pyt

NVIDIA-NeMo/RL#567 · 8 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
[Epic] NeMo-Automodel integration Open

t-pytdensor

NVIDIA-NeMo/RL#578 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Fix multi-turn example convergence Open

bug

NVIDIA-NeMo/RL#593 · 6 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Refactor: separate sync/async vllm Open

UX vllm

NVIDIA-NeMo/RL#599 · 5 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Support for guided decoding in vLLM Open

external r0.6.0 vllm x-futurehouse

NVIDIA-NeMo/RL#603 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Integration between NeMo and NeMo-RL for audio LLMs Open

community-request external needs-design waiting-on-customer x-tii

NVIDIA-NeMo/RL#619 · 8 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
OOM during refit in mcore path when using larger bucket sizes Open

t-mcore vllm

NVIDIA-NeMo/RL#630 · 6 comments · 0 reactions · 2 assignees · Indexed Sep 17, 2026
Sequence packing + no dynamic batching failure on 8b Open

bug t-pytdensor

NVIDIA-NeMo/RL#648 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Qwen2.5-32B w/ sequence_length=16384 OOM on 16nodes H100 using FSDP2 backend Open

bug memory issue t-pytdensor

NVIDIA-NeMo/RL#656 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Add Search-R1 as an example Open

algorithm enhancement

NVIDIA-NeMo/RL#657 · 2 comments · 1 reaction · 1 assignee · Indexed Sep 17, 2026

NVIDIA-NeMo/RL

Get new issues in your inbox