GPU/CPU memory keeps increasing during the TensorRT build phase after adding an OptimizationProfile (memory not reclaimed even after releasing objects).

Description

Environment

TensorRT Version: 10.7

NVIDIA GPU: L40

NVIDIA Driver Version:

CUDA Version:

CUDNN Version:

Operating System:

Python Version (if applicable):

Tensorflow Version (if applicable):

PyTorch Version (if applicable):

Baremetal or Container (if so, version):

Relevant Files

Model link, code and log link:

test_trt_error.cpp

segment_serving_default.onnx.zip

memory_usage_error.log

Steps To Reproduce

Commands or scripts:
build:

g++ -mavx2 -DNDEBUG -DGOOGLE_CUDA -DGOOGLE_TENSORRT -DEIGEN_VECTORIZE_AVX2   -g -O3 -std=c++14   -I./third_party/include/tensorrt-10.7   -I/usr/local/cuda/include   -o test_trt_error test_trt_error.cpp   -L./third_party/lib/tensorrt-10.7   -L/usr/local/cuda/lib64   -lnvonnxparser -lnvinfer -lnvinfer_plugin -lcudart -lnvToolsExt   -lprotobuf -lgflags -lpthread -ldl   -Wl,-rpath,./third_party/lib/tensorrt-10.7 -Wl,-rpath,/usr/local/cuda/lib64

exec:

./test_trt_error &>> memory_usage_error.log &

check log:

#device mem
grep -rn "Free" memory_usage_error.log 
#host mem
grep -rn "Syst" memory_usage_error.log 

Have you tried the latest release?:

Attach the captured .json and .bin files from TensorRT's API Capture tool if you're on an x86_64 Unix system

Can this model run on other frameworks? For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt):

Start with test_trt_error.cpp and the attached segment_serving_default.onnx model, build it with the command in the issue, and inspect memory_usage_error.log using the provided grep commands. Trace the TensorRT build phase involving OptimizationProfile and verify that host and device memory no longer keep increasing after the relevant objects are released.