[Bug]: Degraded output (repetitive "!!!!!" tokens) during FP4 inference on RTX 5090

System Info

trtllm: 1.2.0 rc4

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

When running Qwen3-32B-FP4 on an RTX 5090, the model frequently degenerates into generating repetitive exclamation marks (!!!!!!!!!).
We have verified that the exact same model weights and generation parameters work perfectly fine when using vLLM on the same machine.

trtllm-serve serve /data/models/Qwen3-32B-NVFP4/ --host 0.0.0.0 --port 8201 --backend pytorch --max_batch_size 128 --max_seq_len 16384 --max_num_tokens 4096 --tp_size 2 --pp_size 1 --kv_cache_free_gpu_memory_fraction 0.9 --extra_llm_api_options ifb.yaml

ifb.yaml:

cuda_graph_config: 
  enable_padding: true
  batch_sizes:
    - 1
    - 2
    - 3
    - 4
    - 5
    - 6
    - 7
    - 8
    - 10
    - 12
    - 14
    - 16
    - 20
    - 24
    - 28
    - 32
    - 36
    - 40
    - 48
    - 56
    - 64
    - 72
    - 80
    - 96
    - 112
    - 128
    - 160
    - 192

print_iter_log: false
enable_attention_dp: false
enable_chunked_prefill: true

kv_cache_config:
  enable_block_reuse: false
  dtype: auto

scheduler_config:
  capacity_scheduler_policy: MAX_UTILIZATION

return_perf_metrics: true

Expected behavior

always generate normally

actual behavior

!!!!!!!!

additional notes

None

Before submitting a new issue...

• Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.

Start by reproducing the Qwen3-32B-FP4 inference with the documented trtllm-serve serve command and the provided ifb.yaml, then compare its output with vLLM on the same machine. Focus the investigation on the PyTorch backend with FP4, CUDA graph padding, chunked prefill, and tensor parallelism; done means normal generation instead of repetitive ! tokens.