NVIDIA-NeMo / NVIDIA-NeMo/RL

Wire up TRT-LLM rollout observability metrics (get_logger_metrics/clear_logger_metrics parity with vLLM)

Open
#3,306 0 comments 0 reactions 1 assignee Claimed by @shuyixiong View on GitHub
Feature
Dominant language
Python
Stars
2k
Forks
561
Avg merge
4d 5h
Merged PRs (30d)
145

Description

## Context

During review of #2420, [it was flagged](https://github.com/NVIDIA-NeMo/RL/pull/2420#discussion_r3580086599) that `TrtllmGeneration` doesn't implement rollout observability metrics: `VllmGeneration` overrides `get_logger_metrics` / `clear_logger_metrics` to publish rollout throughput, KV-cache utilization, and in-flight batch counters from AsyncLLM iteration stats, while the TRT-LLM backend has no equivalent plumbing.

#2420 made the gap explicit rather than silent — `TrtllmGeneration.clear_logger_metrics` / `get_logger_metrics` are now explicit no-op/`{}` overrides with docstrings and TODOs ([`trtllm_generation.py`](https://github.com/NVIDIA-NeMo/RL/blob/d3ec162a8ba60133fdcfb90f18227277b4b4ba04/nemo_rl/models/generation/trtllm/trtllm_generation.py#L455-L479)) — but the metrics themselves are still absent: TRT-LLM GRPO runs show none of the rollout telemetry that vLLM runs do.

## Task

- [ ] Fetch TRT-LLM `AsyncLLM` iteration stats from the workers and surface them via `get_logger_metrics`, mirroring `get_vllm_logger_metrics` (rollout throughput, KV-cache utilization, in-flight batch counters — whatever TRT-LLM's stats API exposes)
- [ ] Implement the corresponding `clear_logger_metrics` reset, mirroring `clear_vllm_logger_metrics`
- [ ] Update the TODOs in `trtllm_generation.py` to reference this issue

## References
- Originating thread: https://github.com/NVIDIA-NeMo/RL/pull/2420#discussion_r3580086599
- vLLM reference implementation: `nemo_rl/models/generation/vllm/vllm_generation.py` (`get_logger_metrics`/`clear_logger_metrics` overrides)
- TRT-LLM backend PR: #2420

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.