NVIDIA-NeMo / NVIDIA-NeMo/RL

Move TRT-LLM nightly/release test set to the agreed matrix (mcore slots: non-colocated + async-GRPO)

Open
#3,305 0 comments 0 reactions 2 assignees Claimed by @shuyixiong View on GitHub
Automation Feature
Dominant language
Python
Stars
2k
Forks
561
Avg merge
4d 5h
Merged PRs (30d)
145

Description

## Context

In #2420 review, [the agreed TRT-LLM convergence test matrix](https://github.com/NVIDIA-NeMo/RL/pull/2420#discussion_r3580218996) was:

1. **nightly**: dtensor + colocated + sync
2. **nightly**: mcore + **non-colocated** + **async-GRPO**
3. **release**: nanov3, mcore + **non-colocated** + **async-GRPO**

The PR ships a 2-nightly + 1-release set with the right counts and registrations, but both mcore slots currently run **colocated + sync-GRPO** (all three recipes inherit `colocated.enabled: true` from `examples/configs/grpo_math_1B_trtllm.yaml` and never enable `async_grpo`):

| Slot | Current (in #2420) | Agreed |
|---|---|---|
| nightly 1 | `grpo-qwen3-1.7b-2n4g-fsdp2-trtllm` — dtensor + colocated + sync | ✅ matches |
| nightly 2 | `grpo-qwen2.5-0.5b-1n4g-megatron-trtllm` — mcore + colocated + sync | mcore + non-colocated + async-GRPO |
| release | `grpo-dapomath17k-nanov3-30BA3B-8n4g-megatron-trtllm` — mcore + colocated + sync | nanov3, mcore + non-colocated + async-GRPO |

Non-colocated + async-GRPO is currently exercised only by the functional smoke test (`tests/functional/grpo_trtllm_mcore_non_colocated_async.sh`), not by any convergence-tracked nightly/release recipe — so the deployment mode that stresses NCCL-collective refit and the async trajectory-collector path has no convergence coverage.

## Task

Move the TRT-LLM nightly/release test set from the current all-colocated/sync configuration to the agreed matrix:

- [ ] Flip `grpo-qwen2.5-0.5b-1n4g-megatron-trtllm` (or a successor recipe) to non-colocated + `async_grpo.enabled: true`, with a verified convergence run
- [ ] Flip the release `grpo-dapomath17k-nanov3-30BA3B-8n4g-megatron-trtllm` to non-colocated + `async_grpo.enabled: true`, with a verified convergence run
- [ ] Update the recipe names if the axes change (naming convention encodes the config), and re-register in `tests/test_suites/nightly_gb200.txt` / `release_gb200.txt`
- [ ] Attach convergence curves for the updated runs

## References
- Agreed matrix + discussion: https://github.com/NVIDIA-NeMo/RL/pull/2420#discussion_r3580218996
- TRT-LLM backend PR: #2420

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.