Memory leaky during the Polygraphy execution process. (mark all)

Description

I am using Polygraphy to evaluate the accuracy of a super-resolution model. However, when I use "mark all" option, the memory quickly increases and fills up all my available memory (over 60GB) within a short period of time. How can I resolve this issue? Thank you very much. If needed, I can provide a test ONNX model of around 150MB.

Environment

TensorRT Version:
nvcr.io/nvidia/tensorrt 23.04-py3
NVIDIA GPU:
3060
NVIDIA Driver Version:
12
CUDA Version:
nvcr.io/nvidia/tensorrt 23.04-py3
CUDNN Version:
nvcr.io/nvidia/tensorrt 23.04-py3

Commands or scripts:

polygraphy run static_fp32_18.onnx \
    --onnxrt --trt \
    --atol 1e-1 --rtol 1e-3 \
    --verbose \
    --onnx-outputs mark all \
    --trt-outputs mark all \
    --input-shapes 'input:[1,1,400,400]' \
    > result-run-FP32-MarkAll.txt

Can this model run on other frameworks? For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt):

yes , It can run successfully on onnxruntime.