Different output with same input between dynamic and static shape model
@kevinch-nv is already working on this.
Since Nov 22, 2024.
- Dominant language
- C++
- Stars
- 13.4k
- Forks
- 2.4k
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 2
Description
Description
I have onnx model with fixed input from files. I try to run my model with trtexec by command:
trtexec with dynamic shape
/usr/src/tensorrt/bin/trtexec --onnx=~/model.onnx --loadInputs=descriptors_0:/home/pavel/test/descr0.dat,descriptors_1:/home/pavel/test/descr1.dat --minShapes=descriptors_0:1x256x1,descriptors_1:1x256x1 --optShapes=descriptors_0:1x256x1000,descriptors_1:1x256x1000 --maxShapes=descriptors_0:1x256x1024,descriptors_1:1x256x1024 --shapes=descriptors_0:1x256x100,descriptors_1:1x256x5 --exportOutput=/home/pavel/dynamic_debug.json --saveEngine=/home/pavel/trt.engine
I receive unexpected output from dynamic_debug.json (many numbers is zero) and try to check same onnx model with static shape by command:
trtexec with static shape
/usr/src/tensorrt/bin/trtexec --onnx=~/model.onnx --loadInputs=descriptors_0:/home/pavel/test/descr0.dat,descriptors_1:/home/pavel/test/descr1.dat --shapes=descriptors_0:1x256x100,descriptors_1:1x256x5 --exportOutput=/home/pavel/static_debug.json --saveEngine=/home/pavel/trt.engine
In this case results looks good.
How can I receive same output with dynamic shape model?
Environment
TensorRT Version: 10.6.0.26
NVIDIA GPU: NVIDIA GeForce GTX 1650 Ti
NVIDIA Driver Version: 550.127.05
CUDA Version: 12.4
CUDNN Version: 9.5.1
Operating System: Kubuntu 24.04
Relevant Files
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.