NVIDIA / NVIDIA/TensorRT

Regression of VRAM consumption and performance on RIFE 4.25 model starting from TensorRT 10.14

Open
#4,721 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Module:Performance
Dominant language
C++
Stars
13.4k
Forks
2.4k
Avg merge
5d 3h
Merged PRs (30d)
2

Description

Description

With FP16 dynamic engines generated from same onnx model file, the GPU memory usage and performance while inferencing are severely regressed in new TensorRT versions:

TensorRT version 10.13.3.9 10.15.1.29
VRAM usage (monitored from task manager) ~2.7G ~10G
Mean GPU Compute Time (trtexec inference) 3.53ms 9.59ms

After more testing, this issue firstly appeared in 10.14.

Environment

TensorRT Version: 10.15.1.29

NVIDIA GPU: GeForce RTX 5090 D

NVIDIA Driver Version: 595.79

CUDA Version: 13.2

CUDNN Version: 9.20

Operating System: Windows 11 25H2 26000.8037

Python Version (if applicable): N/A

Tensorflow Version (if applicable): N/A

PyTorch Version (if applicable): N/A

Baremetal or Container (if so, version): Baremetal Windows, no containers involved

Relevant Files

Model link:
rife_v4.25, the one in rife_v2

Steps To Reproduce

Commands or scripts:
Generate engine:

trtexec --onnx=C:/Users/datasone/AppData/Roaming/VapourSynth/plugins64\models\rife_v2\rife_v4.25.onnx --saveEngine=C:/Users/datasone/work/temp/10.15.engine --minShapes=input:1x7x256x320 --optShapes=input:1x7x1080x1920 --maxShapes=input:1x7x2160x4096 --fp16 --tacticSources=+CUBLAS,+CUBLAS_LT,+CUDNN,+EDGE_MASK_CONVOLUTIONS,+JIT_CONVOLUTIONS --useCudaGraph --noDataTransfers --noTF32 --inputIOFormats=fp16:chw --outputIOFormats=fp16:chw --builderOptimizationLevel=3 --precisionConstraints=obey --layerPrecisions=/Cast_2:fp32,/Cast_3:fp32,/Cast_5:fp32,/Cast_7:fp32,/Reciprocal:fp32,/Reciprocal_1:fp32,/Mul:fp32,/Mul_1:fp32,/Mul_8:fp32,/Mul_10:fp32,/Sub_5:fp32,/Sub_6:fp32,ONNXTRT_Broadcast_236:fp32,ONNXTRT_Broadcast_238:fp32,ONNXTRT_Broadcast_273:fp32,ONNXTRT_Broadcast_275:fp32,ONNXTRT_Broadcast_*:fp32 --verbose --useSpinWait --exportLayerInfo=C:/Users/datasone/work/temp/10.15.graph.json --exportProfile=C:/Users/datasone/work/temp/10.15.profile.json --profilingVerbosity=detailed >trtexec.10.15.log

Inference test:

trtexec --loadEngine=10.15.engine --shapes=input:1x7x1080x1920 --noDataTransfers --useSpinWait --warmUp=500 --duration=5

The following files are generated from the command above

trtexec.10.13.3.log
10.13.3.graph.json
10.13.3.profile.json

trtexec.10.15.log
10.15.graph.json
10.15.profile.json

Have you tried the latest release?: Yes

Attach the captured .json and .bin files from TensorRT's API Capture tool if you're on an x86_64 Unix system

Can this model run on other frameworks? For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt): N/A

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the reported comparison with the RIFE v4.25 ONNX model using the provided trtexec engine-generation and inference commands. Compare the attached 10.13 and 10.15 graph, profile, and log files to identify what changed starting in TensorRT 10.14; done means explaining or resolving the VRAM and GPU-time regression.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.