Wire up TRT-LLM rollout observability metrics (get_logger_metrics/clear_logger_metrics parity with vLLM)
- Dominant language
- Python
- Stars
- 2k
- Forks
- 561
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 145
Description
## Context
During review of #2420, [it was flagged](https://github.com/NVIDIA-NeMo/RL/pull/2420#discussion_r3580086599) that `TrtllmGeneration` doesn't implement rollout observability metrics: `VllmGeneration` overrides `get_logger_metrics` / `clear_logger_metrics` to publish rollout throughput, KV-cache utilization, and in-flight batch counters from AsyncLLM iteration stats, while the TRT-LLM backend has no equivalent plumbing.
#2420 made the gap explicit rather than silent — `TrtllmGeneration.clear_logger_metrics` / `get_logger_metrics` are now explicit no-op/`{}` overrides with docstrings and TODOs ([`trtllm_generation.py`](https://github.com/NVIDIA-NeMo/RL/blob/d3ec162a8ba60133fdcfb90f18227277b4b4ba04/nemo_rl/models/generation/trtllm/trtllm_generation.py#L455-L479)) — but the metrics themselves are still absent: TRT-LLM GRPO runs show none of the rollout telemetry that vLLM runs do.
## Task
- [ ] Fetch TRT-LLM `AsyncLLM` iteration stats from the workers and surface them via `get_logger_metrics`, mirroring `get_vllm_logger_metrics` (rollout throughput, KV-cache utilization, in-flight batch counters — whatever TRT-LLM's stats API exposes)
- [ ] Implement the corresponding `clear_logger_metrics` reset, mirroring `clear_vllm_logger_metrics`
- [ ] Update the TODOs in `trtllm_generation.py` to reference this issue
## References
- Originating thread: https://github.com/NVIDIA-NeMo/RL/pull/2420#discussion_r3580086599
- vLLM reference implementation: `nemo_rl/models/generation/vllm/vllm_generation.py` (`get_logger_metrics`/`clear_logger_metrics` overrides)
- TRT-LLM backend PR: #2420
Contributor guide
Assessment
This issue has not been assessed yet.