[Performance] riva-translate-4b latency is 35.3% above reference

This was generated by AI during triage.

Summary

riva-translate-4b completes aligned FP16 generation and matches exact token IDs, but TRTMC latency is 35.3% above the reference.

Observed behavior

• Precision: TRTMC FP16, reference FP16
• Output contract: exact token IDs (matched)
• Request: batch 1, 20 new tokens
• Measurement: 3 warmups followed by 10 iterations
• Candidate p50: 397.75 ms
• Reference p50: 293.93 ms
• Regression: +35.3% (allowed margin: 5%)

Expected behavior

The translation generation path should remain within the registered performance margin.

Acceptance criteria

• Profile prefill, decode, KV-cache handling, and synchronization.
• Identify and fix the dominant latency source.
• Preserve exact-token output and the existing performance gate.

Reproduction platform

• Hardware: NVIDIA Thor X
• Architecture: aarch64
• TensorRT: 11.0
• GPU carveout: 48 GiB

Reproduce the aligned FP16 generation benchmark on NVIDIA Thor X with batch 1, 20 new tokens, three warmups, and ten iterations. Profile prefill, decode, KV-cache handling, and synchronization to identify the dominant latency source; done means latency is within the 5% margin while exact token IDs and the existing performance gate remain unchanged.