User allocator error allocating 86114304000-byte buffer failure of TensorRT 10.6 when running demo_img2vid. py on GPU rtx4090

Description

I tried to convert onnx to an engine file using demo_img2vid. py on rtx4090，but received the following error message：

[E] [defaultAllocator.cpp::allocate::31] Error Code 1: Cuda Runtime (out of memory)

[E] Error Code: 9: Skipping tactic 0x0000000000000000 due to exception [tunable_graph.cpp:create:117] autotuning: User allocator error allocating 86114304000-byte buffer

[E] [virtualMemoryBuffer.cpp::resizePhysical::154] Error Code 2: OutOfMemory (Requested size was 16581132288 bytes.)

[E] [virtualMemoryBuffer.cpp::resizePhysical::154] Error Code 2: OutOfMemory (Requested size was 16157507584 bytes.)

[E] [defaultAllocator.cpp::allocate::31] Error Code 1: Cuda Runtime (out of memory)

[E] Error Code: 9: Skipping tactic 0x0000000000000000 due to exception [tunable_graph.cpp:create:117] autotuning: User allocator error allocating 86114304000-byte buffer

[E] [virtualMemoryBuffer.cpp::resizePhysical::154] Error Code 2: OutOfMemory (Requested size was 16286482432 bytes.)

[E] [virtualMemoryBuffer.cpp::resizePhysical::154] Error Code 2: OutOfMemory (Requested size was 15862857728 bytes.)

[E] [defaultAllocator.cpp::allocate::31] Error Code 1: Cuda Runtime (out of memory)

[E] Error Code: 9: Skipping tactic 0x0000000000000000 due to exception [tunable_graph.cpp:create:117] autotuning: User allocator error allocating 11206656000-byte buffer

[E] [defaultAllocator.cpp::allocate::31] Error Code 1: Cuda Runtime (out of memory)

[E] Error Code: 9: Skipping tactic 0x0000000000000000 due to exception [tunable_graph.cpp:create:117] autotuning: User allocator error allocating 11206656000-byte buffer

[E] [virtualMemoryBuffer.cpp::resizePhysical::154] Error Code 2: OutOfMemory (Requested size was 12238979072 bytes.)

[E] [virtualMemoryBuffer.cpp::resizePhysical::154] Error Code 2: OutOfMemory (Requested size was 12091129856 bytes.)

[E] [virtualMemoryBuffer.cpp::resizePhysical::154] Error Code 2: OutOfMemory (Requested size was 12681478144 bytes.)

[E] [virtualMemoryBuffer.cpp::resizePhysical::154] Error Code 2: OutOfMemory (Requested size was 12460228608 bytes.)

[E] [virtualMemoryBuffer.cpp::resizePhysical::154] Error Code 2: OutOfMemory (Requested size was 12091129856 bytes.)

[E] [defaultAllocator.cpp::allocate::31] Error Code 1: Cuda Runtime (out of memory)

[E] Error Code: 9: Skipping tactic 0x0000000000000000 due to exception [tunable_graph.cpp:create:117] autotuning: User allocator error allocating 11206656000-byte buffer

[E] [defaultAllocator.cpp::allocate::31] Error Code 1: Cuda Runtime (out of memory)

[E] Error Code: 9: Skipping tactic 0x0000000000000000 due to exception [tunable_graph.cpp:create:117] autotuning: User allocator error allocating 11206656000-byte buffer

[E] [defaultAllocator.cpp::allocate::31] Error Code 1: Cuda Runtime (out of memory)

[E] Error Code: 9: Skipping tactic 0x0000000000000000 due to exception [tunable_graph.cpp:create:117] autotuning: User allocator error allocating 11206656000-byte buffer

[E] [defaultAllocator.cpp::allocate::31] Error Code 1: Cuda Runtime (out of memory)

[E] Error Code: 9: Skipping tactic 0x0000000000000000 due to exception [tunable_graph.cpp:create:117] autotuning: User allocator error allocating 86114304000-byte buffer

[E] [virtualMemoryBuffer.cpp::resizePhysical::154] Error Code 2: OutOfMemory (Requested size was 17466130432 bytes.)

[E] [virtualMemoryBuffer.cpp::resizePhysical::154] Error Code 2: OutOfMemory (Requested size was 17042505728 bytes.)

[E] [defaultAllocator.cpp::allocate::31] Error Code 1: Cuda Runtime (out of memory)

[E] Error Code: 9: Skipping tactic 0x0000000000000000 due to exception [tunable_graph.cpp:create:117] autotuning: User allocator error allocating 86114304000-byte buffer

[E] [virtualMemoryBuffer.cpp::resizePhysical::154] Error Code 2: OutOfMemory (Requested size was 17466130432 bytes.)

[E] [virtualMemoryBuffer.cpp::resizePhysical::154] Error Code 2: OutOfMemory (Requested size was 17042505728 bytes.)

[E] [defaultAllocator.cpp::allocate::31] Error Code 1: Cuda Runtime (out of memory)

[E] Error Code: 9: Skipping tactic 0x0000000000000000 due to exception [tunable_graph.cpp:create:117] autotuning: User allocator error allocating 86114304000-byte buffer

[E] [virtualMemoryBuffer.cpp::resizePhysical::154] Error Code 2: OutOfMemory (Requested size was 16286482432 bytes.)

How should I fix this error？

Environment

docker：nvcr.io/nvidia/pytorch:24.07-py3

TensorRT Version: 10.6

NVIDIA GPU: rtx4090

NVIDIA Driver Version:NVIDIA-SMI 535.161.07

CUDA Version:12.2

CUDNN Version:

Operating System:

Python Version (if applicable): 3.10

Tensorflow Version (if applicable):

PyTorch Version (if applicable):2.4.0a0+3bcc3cddb5.nv24.7

Baremetal or Container (if so, version):

Steps To Reproduce

docker run --rm -it --gpus all -v $PWD:/workspace nvcr.io/nvidia/pytorch:24.07-py3 /bin/bash
python3 -m pip install --upgrade pip
pip install --pre tensorrt-cu12
export TRT_OSSPATH=/workspace
cd $TRT_OSSPATH/demo/Diffusion
pip3 install -r requirements.txt
python3 demo_img2vid.py --version svd-xt-1.1 --onnx-dir onnx-svd-xt-1-1 --engine-dir engine-svd-xt-1-1 --hf-token=$HF_TOKEN

Start with demo/Diffusion/demo_img2vid.py and reproduce the command in the reported nvcr.io/nvidia/pytorch:24.07-py3 environment on an RTX 4090. Compare the TensorRT 10.6 engine-building behavior and GPU memory requests with the stated versions; the issue is complete only when a supported configuration or documented resolution is established.