Move TRT-LLM nightly/release test set to the agreed matrix (mcore slots: non-colocated + async-GRPO)
- Dominant language
- Python
- Stars
- 2k
- Forks
- 561
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 145
Description
## Context
In #2420 review, [the agreed TRT-LLM convergence test matrix](https://github.com/NVIDIA-NeMo/RL/pull/2420#discussion_r3580218996) was:
1. **nightly**: dtensor + colocated + sync
2. **nightly**: mcore + **non-colocated** + **async-GRPO**
3. **release**: nanov3, mcore + **non-colocated** + **async-GRPO**
The PR ships a 2-nightly + 1-release set with the right counts and registrations, but both mcore slots currently run **colocated + sync-GRPO** (all three recipes inherit `colocated.enabled: true` from `examples/configs/grpo_math_1B_trtllm.yaml` and never enable `async_grpo`):
| Slot | Current (in #2420) | Agreed |
|---|---|---|
| nightly 1 | `grpo-qwen3-1.7b-2n4g-fsdp2-trtllm` — dtensor + colocated + sync | ✅ matches |
| nightly 2 | `grpo-qwen2.5-0.5b-1n4g-megatron-trtllm` — mcore + colocated + sync | mcore + non-colocated + async-GRPO |
| release | `grpo-dapomath17k-nanov3-30BA3B-8n4g-megatron-trtllm` — mcore + colocated + sync | nanov3, mcore + non-colocated + async-GRPO |
Non-colocated + async-GRPO is currently exercised only by the functional smoke test (`tests/functional/grpo_trtllm_mcore_non_colocated_async.sh`), not by any convergence-tracked nightly/release recipe — so the deployment mode that stresses NCCL-collective refit and the async trajectory-collector path has no convergence coverage.
## Task
Move the TRT-LLM nightly/release test set from the current all-colocated/sync configuration to the agreed matrix:
- [ ] Flip `grpo-qwen2.5-0.5b-1n4g-megatron-trtllm` (or a successor recipe) to non-colocated + `async_grpo.enabled: true`, with a verified convergence run
- [ ] Flip the release `grpo-dapomath17k-nanov3-30BA3B-8n4g-megatron-trtllm` to non-colocated + `async_grpo.enabled: true`, with a verified convergence run
- [ ] Update the recipe names if the axes change (naming convention encodes the config), and re-register in `tests/test_suites/nightly_gb200.txt` / `release_gb200.txt`
- [ ] Attach convergence curves for the updated runs
## References
- Agreed matrix + discussion: https://github.com/NVIDIA-NeMo/RL/pull/2420#discussion_r3580218996
- TRT-LLM backend PR: #2420
Contributor guide
Assessment
This issue has not been assessed yet.