NVIDIA-NeMo / NVIDIA-NeMo/Automodel
RL + dLLM
Open
@zyzhou5 is already working on this.
Since Aug 27, 2026.
dLLM
PoR
- Dominant language
- Python
- Stars
- 963
- Forks
- 318
- Avg merge
- 3d 20h
- Merged PRs (30d)
- 143
Description
BlockjustGRPO-fast + nemotron-lab-diffusion + AM + RL
AM integration Plan
https://docs.google.com/document/d/1xC7WKKNLlI5hEIly9--WelhO92NYKbgBMCqo62Po_1s/edit?usp=sharing
draft PR: https://github.com/zyzhou5/RL/pull/4
**Plan **
- Working e2e pipeline with AM backend (based on Sajad's current design https://github.com/sajadn/diffusion_RL/commits/dllm_clean/). Planning to do the following test
- model loading
- model forward/backward
- loss parity (algorithm)
- vLLM supports
- 100 steps of training and compare reward
- Other features
- CP
- Other algorithms
- JustGRPO
- …
Other things
- Sajad mentioned about a new design of the dLLM RL pipeline in Nemo-RL (https://docs.google.com/document/d/1HHzRpLfsRalQkqLGFe0vljZjRO7dwKaKWH43jZPxCu4/edit?tab=t.0), we will need to refactor on the AM side as well once that new design is implemented in Nemo-RL
- Timeline - might happen in a few weeks
- Check if huiying's refactor on AM + Nemo-RL will cause any issue NVIDIA-NeMo/Automodel#3614
- Timeline - estimated to be done by the end of September
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.