[Bug]: Suspicious memory leak in process "/usr/bin/python -R -m mpi4py.futures.server"

System Info

System Info
GPU: H20 96G * 8
Environment: 1.2.0.rc1 and 1.0.0.rc4, both versions have memory leak problem, it seems not related to a specific version
Model: Qwen series, we tested Qwen3-8B, Qwen3-32B and QwQ-32B, it is not related to a specific model.

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

Keep sending requests and observe the memory usage, we can see the memory usage will continue to grow.
When test started：
For example, we tested Qwen3-8B with continuous serial transmission of requests "Write a short essay of 1000 words."

After 12 hours：

Expected behavior

No memory leak

actual behavior

Some online services （Qwen2.5-7B / QwQ-32B）have already triggered OOM (Out of Memory) errors, causing process crashes.

additional notes

We already tried [https://github.com/NVIDIA/TensorRT-LLM/issues/6901][Fix]: Breaking Change: disable nvtx annotation by default. However, this fix doesn't seem to have worked.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.