[Performance] glm-4-9b latency is 25.5% above reference

This was generated by AI during triage.

Summary

glm-4-9b completes aligned FP16 generation and matches exact token IDs, but TRTMC latency is 25.5% above the reference.

Observed behavior

• Precision: TRTMC FP16, reference FP16
• Output contract: exact token IDs (matched)
• Request: batch 1, 20 new tokens
• Measurement: 3 warmups followed by 10 iterations
• Candidate p50: 1,807.21 ms
• Reference p50: 1,440.22 ms
• Regression: +25.5% (allowed margin: 5%)

Expected behavior

The generation path should remain within the registered performance margin after warmup.

Acceptance criteria

• Profile prefill, decode, KV-cache access, and synchronization.
• Identify and fix the dominant bottleneck.
• Preserve exact-token output and the existing performance gate.

Reproduction platform

• Hardware: NVIDIA Thor X
• Architecture: aarch64
• TensorRT: 11.0
• GPU carveout: 48 GiB

No file, test, or entry point is named. Start by reproducing the batch-1, 20-token FP16 measurement on NVIDIA Thor, then profile prefill, decode, KV-cache access, and synchronization against the reference. Done means the dominant bottleneck is addressed, exact token IDs remain unchanged, and the existing performance gate is within its 5% margin.