After upgrading from 8.6 to 10.8 or 10.9, tensorrt's results are inconsistent with onnxrt
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 13.4k
- Forks
- 2.4k
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 2
Description
Description
After upgrading tensorrt to 10.8, the model accuracy decreased.
After setting all nodes of the model to output, the model accuracy was aligned. It was suspected that the fusion strategy introduced by the upgrade caused the accuracy problem.
Finally, we used the polygraphy tool to find an onnx subgraph that could reproduce the problem.
By the way, it needs to be reminded that the model only uses fp32 precision, not even fp16 or int8
Environment
TensorRT Version: 10.8 & 10.9
NVIDIA GPU: 3090
NVIDIA Driver Version: 550.67
CUDA Version: 12.2
Relevant Files
I post onnx file here:
Model link: https://github.com/2730gf/issues/blob/main/trt_inconsistent/mini_graph.onnx
Steps To Reproduce
Commands or scripts:
polygraphy run mini_graph.onnx -v -v -v -v -v --pool-limit workspace:20G --onnxrt --trt --validate --atol 1e-4 --rtol 1e-3 --onnx-outputs p2o.Concat.125 --trt-outputs p2o.Concat.125
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked mini_graph.onnx and run the supplied Polygraphy command using TensorRT and ONNX Runtime. Compare the outputs at p2o.Concat.125 under TensorRT 10.8/10.9; done means the precision discrepancy is reproduced and its TensorRT fusion cause is identified or corrected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100