Error: No such command 'optimize' when running `nat optimize` + TTFT metric reproduction issues

How would you describe the priority of this documentation request

High

Please provide a link or source to the relevant docs

https://github.com/NVIDIA/NeMo-Agent-Toolkit/blob/develop/examples/dynamo_integration/react_benchmark_agent/README.md

Describe the problems in the documentation

Issue Description

Hi NVIDIA NeMo Dynamo team,
I am following the official Dynamo integration React benchmark tutorial:
https://github.com/NVIDIA/NeMo-Agent-Toolkit/tree/develop/examples/dynamo_integration/react_benchmark_agent
and I run into some problems during reproduction and benchmark testing.

1. Command Execution Error

When I run the command:

nat optimize --config_file examples/dynamo_integration/react_benchmark_agent/configs/optimize_rethinking_full_test.yml

I get the error:
Error: No such command 'optimize'.

I wonder if this command has been renamed/removed in the current develop branch, or if I am missing extra installation/setup steps.

---

2. Thompson Router Benchmark Result

Additionally, I followed the router part of the blog:
https://developer.nvidia.com/blog/full-stack-optimizations-for-agentic-inference-with-nvidia-dynamo/#layer-2-the-router

I ran the benchmark in https://github.com/NVIDIA/NeMo-Agent-Toolkit/tree/develop/examples/dynamo_integration

Performance Results

I have attached my benchmark comparison screenshot below.

• Left side: without Thompson Sampling Router(Baseline)

• Right side: with Thompson Sampling Router enabled

• In no_rethinking_full_test: only p50 TTFT is reduced and TPS is improved. All other TTFT metrics (mean, p90, p95, p99) are higher than the baseline without Thompson Router.

• In rethinking_full_test: there is no positive optimization observed on any latency or throughput metrics at all; instead, we see a slight performance degradation after enabling the Thompson router.

My Environment

• GPU: 8× H20
• Model: llama-3.3-70b-Instruct
• Config: TP=2, 2 workers, 4 GPUs total, along with other default benchmark parameters

Questions

1. Guidance to resolve the missing nat optimize command error would be greatly appreciated.
2. Are the router performance results published in the blog obtained under this default TP=2 / 2 worker configuration?
3. Why does Thompson Router only improve median TTFT, while heavily degrading mean and long-tail TTFT latency?
4. What parameters, tuning suggestions or environment adjustments can I apply to fully reproduce the blog's oerall latency optimization results?
   Thanks very much for your help! 🙏

(Optional) Propose a correction

No response

Code of Conduct

• I agree to follow this project's Code of Conduct
• I have searched the open documentation issues and have found no duplicates for this bug report

Start with examples/dynamo_integration/react_benchmark_agent/README.md and the referenced optimize_rethinking_full_test.yml, then run the documented nat optimize command and compare it with the current develop branch. Review the linked Dynamo integration example and benchmark results to establish the expected setup, parameters, and TTFT behavior. Done means the documentation explains the command availability, reproduction environment, and interpretation of the router metrics.