[Bug]: Memory leak: Qwen3 30B A3B pytorch backend
Open
@hchings is already working on this.
Since May 12, 2026.
bug
Memory
Pytorch
- Dominant language
- Python
- Stars
- 14.7k
- Forks
- 2.8k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 489
Description
System Info
- CPU architecture: amd64
- CPU memory size: 80Gb
- GPU properties
- GPU name: NVIDIA H200
- GPU memory size: 71424MiB (4g.71gb MIG)
- Driver Version: 580.126.20
- CUDA Version: 13.1
- TensorRT-LLM docker image used: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc12
Who can help?
No response
Information
- The official example scripts
- My own modified scripts
Tasks
- An officially supported task in the
examplesfolder (such as GLUE/SQuAD, ...) - My own task or dataset (give details below)
Reproduction
Steps to reproduce behaviour:
cat <<'EOF' > config.yaml
max_seq_len: 32768
max_num_tokens: 16384
max_batch_size: 64
num_postprocess_workers: 4
stream_interval: 5
enable_chunked_prefill: true
return_perf_metrics: true
enable_iter_perf_stats: true
enable_iter_req_stats: true
cuda_graph_config:
enable_padding: true
max_batch_size: 32
EOF
trtllm-serve Qwen/Qwen3-30B-A3B-Instruct-2507-FP8 --config=config.yamlfor i in $(seq 1 100000); do curl -s -X POST http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{"messages":[{"role":"system","content":"You are a helpful assistant."},{"role":"user","content":"Say '\''ok'\''"}],"model":"dsp.optimized.qwen3-30b-a3b","max_tokens":512,"temperature":1e-05}'; echo; done
Expected behavior
Container memory (RAM) usage shouldn't keep increasing as requests are served.
actual behavior
Container memory usage keeps growing as requests are served till it's OOMKilled.
additional notes
- The memory usage remains flat when there is no traffic.
- This happens with 1.3.0rc13 as well.
Before submitting a new issue...
- Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.