Using max_attention_window (VSWA) reduces concurrent batch size and causes drop in throughput (gemma3 trt backend)

System Info

GPU: NVIDIA A100, NVIDIA H100
TensorRT-LLM version: 1.0.0rc5
TensorRT-LLM commit: b3ca159787226d59717474a304041d9dcb27dd42

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

Build: following gemma3 guide

Serve: trtllm-serve with max_attention_window = [512, 512, 512, 512, 512, 3100]

Expected behavior

When the sequence length is shorter than the minimum attention window,
concurrent batch size remains the same compared to when max_attention_window is not used.
When it is longer, the batch size increases.
It works well in vLLM.

actual behavior

However the batch size decreases in both cases, resulting in a significant drop in throughput.

additional notes

This behavior has continued since the referenced commit.

Start by reproducing the Gemma3 setup from the examples/models/core/gemma guide with trtllm-serve and the reported max_attention_window values. Compare concurrent batch size and throughput for shorter and longer sequences, then inspect the behavior introduced by the referenced commit in pull request 4667. Done means the configured window no longer reduces batch size for shorter sequences and improves it for longer ones.