❓ [Question] Excessive GPU memory usage during constant folding pass causes OOM (Torch-TensorRT + Qwen-Image)

❓ Question

When compiling and running the Qwen-Image model with Torch-TensorRT, the peak GPU memory usage exceeds 95GB, resulting in CUDA out-of-memory (OOM) errors.

For comparison, using the Inductor backend only requires around 64GB peak VRAM. After debugging, the issue appears to originate primarily from the constant folding pass:

https://github.com/pytorch/TensorRT/blob/08bfca206ca47da4a4f217072124a6840f451511/py/torch_tensorrt/dynamo/lowering/passes/constant_folding.py#L22-L65

What you have already tried

Setting offload_module_to_cpu=True successfully bypasses the problematic constant folding behavior and avoids the GPU OOM. However, this workaround significantly increases host (system RAM) memory consumption to an unacceptable level.

Environment

• PyTorch Version：2.9
• CPU Architecture: x86
• OS (e.g., Linux): Linux
• Python version: 3.12
• CUDA version: 12.9

Are there any better / recommended ways to reduce peak GPU memory (or host memory) usage during the lowering / compilation phase?

Is there any plan in future releases to optimize the constant folding pass for lower memory footprint?

Any suggestions or insights would be greatly appreciated!