[Performance] qwen3-4b-instruct-2507 latency is 39.7% above reference

This was generated by AI during triage.

Summary

qwen3-4b-instruct-2507 completes aligned FP16 generation and matches exact token IDs, but TRTMC latency is 39.7% above the reference.

Observed behavior

• Precision: TRTMC FP16, reference FP16
• Output contract: exact token IDs (matched)
• Request: batch 1, 10 new tokens
• Measurement: 3 warmups followed by 10 iterations
• Candidate p50: 153.40 ms
• Reference p50: 109.81 ms
• Regression: +39.7% (allowed margin: 5%)

Expected behavior

The generation path should remain within the registered performance margin after warmup.

Acceptance criteria

• Profile prefill, decode, KV-cache access, and synchronization.
• Identify and fix the dominant bottleneck.
• Preserve exact-token output and the existing performance gate.

Reproduction platform

• Hardware: NVIDIA Thor X
• Architecture: aarch64
• TensorRT: 11.0
• GPU carveout: 48 GiB

Start by profiling the generation path on NVIDIA Thor, separating prefill, decode, KV-cache access, and synchronization as described in the issue. Compare the candidate and reference measurements, then verify that the dominant bottleneck is addressed, exact token IDs remain unchanged, and latency returns within the 5% performance margin.