Trt-engine is 2x slower than the onnx model
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 13.4k
- Forks
- 2.4k
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 2
Description
Description
I tried to inference the sam2-decoder with trt-engine, but, as you can see from the screenshots below, it is much slower than the inference with onnx model.
TRT
ORT
For better investigating, the onnx model is in attachment.
Any clue would be appreciated, thanks in advance!
Environment
TensorRT Version: 10.3.0
NVIDIA GPU: Jetson Orin 64GB
NVIDIA Driver Version: 540.4.0
CUDA Version: 12.6
CUDNN Version: 9.3.0
Operating System:
Python Version (if applicable): 3.10.12
Tensorflow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if so, version):
Relevant Files
Model link:
Model for Debugging
Steps To Reproduce
Commands or scripts:
Have you tried the latest release?:
Can this model run on other frameworks? For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt):
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The payload names only the attached ONNX model and the sam2-decoder; no repository file, test, or runnable command is provided. Start by reproducing the TensorRT and ONNXRuntime comparison in the listed environment, then document the cause and verify that TensorRT performance is comparable under the same conditions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100