Regression of VRAM consumption and performance on RIFE 4.25 model starting from TensorRT 10.14
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 13.4k
- Forks
- 2.4k
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 2
Description
Description
With FP16 dynamic engines generated from same onnx model file, the GPU memory usage and performance while inferencing are severely regressed in new TensorRT versions:
| TensorRT version | 10.13.3.9 | 10.15.1.29 |
|---|---|---|
| VRAM usage (monitored from task manager) | ~2.7G | ~10G |
| Mean GPU Compute Time (trtexec inference) | 3.53ms | 9.59ms |
After more testing, this issue firstly appeared in 10.14.
Environment
TensorRT Version: 10.15.1.29
NVIDIA GPU: GeForce RTX 5090 D
NVIDIA Driver Version: 595.79
CUDA Version: 13.2
CUDNN Version: 9.20
Operating System: Windows 11 25H2 26000.8037
Python Version (if applicable): N/A
Tensorflow Version (if applicable): N/A
PyTorch Version (if applicable): N/A
Baremetal or Container (if so, version): Baremetal Windows, no containers involved
Relevant Files
Model link:
rife_v4.25, the one in rife_v2
Steps To Reproduce
Commands or scripts:
Generate engine:
trtexec --onnx=C:/Users/datasone/AppData/Roaming/VapourSynth/plugins64\models\rife_v2\rife_v4.25.onnx --saveEngine=C:/Users/datasone/work/temp/10.15.engine --minShapes=input:1x7x256x320 --optShapes=input:1x7x1080x1920 --maxShapes=input:1x7x2160x4096 --fp16 --tacticSources=+CUBLAS,+CUBLAS_LT,+CUDNN,+EDGE_MASK_CONVOLUTIONS,+JIT_CONVOLUTIONS --useCudaGraph --noDataTransfers --noTF32 --inputIOFormats=fp16:chw --outputIOFormats=fp16:chw --builderOptimizationLevel=3 --precisionConstraints=obey --layerPrecisions=/Cast_2:fp32,/Cast_3:fp32,/Cast_5:fp32,/Cast_7:fp32,/Reciprocal:fp32,/Reciprocal_1:fp32,/Mul:fp32,/Mul_1:fp32,/Mul_8:fp32,/Mul_10:fp32,/Sub_5:fp32,/Sub_6:fp32,ONNXTRT_Broadcast_236:fp32,ONNXTRT_Broadcast_238:fp32,ONNXTRT_Broadcast_273:fp32,ONNXTRT_Broadcast_275:fp32,ONNXTRT_Broadcast_*:fp32 --verbose --useSpinWait --exportLayerInfo=C:/Users/datasone/work/temp/10.15.graph.json --exportProfile=C:/Users/datasone/work/temp/10.15.profile.json --profilingVerbosity=detailed >trtexec.10.15.log
Inference test:
trtexec --loadEngine=10.15.engine --shapes=input:1x7x1080x1920 --noDataTransfers --useSpinWait --warmUp=500 --duration=5
The following files are generated from the command above
trtexec.10.13.3.log
10.13.3.graph.json
10.13.3.profile.json
trtexec.10.15.log
10.15.graph.json
10.15.profile.json
Have you tried the latest release?: Yes
Attach the captured .json and .bin files from TensorRT's API Capture tool if you're on an x86_64 Unix system
Can this model run on other frameworks? For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt): N/A
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the reported comparison with the RIFE v4.25 ONNX model using the provided trtexec engine-generation and inference commands. Compare the attached 10.13 and 10.15 graph, profile, and log files to identify what changed starting in TensorRT 10.14; done means explaining or resolving the VRAM and GPU-time regression.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100