microsoft / microsoft/onnxruntime
[Performance] NV TRT RTX provider performance slower than TensorRT on RTX 4000 Ada
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the issue
When running inference using the NV TRT RTX execution provider in ONNX Runtime on an RTX 4000 Ada GPU, I observed that it is slower than the TensorRT execution provider for the same model.
**Performance observation**
- TensorRT provider inference time: 40 seconds
- NV TRT RTX provider inference time: 60 seconds
**Expected behavior**
NV TRT RTX execution provider should perform as well as or better than TensorRT provider on the same hardware and model.
Please let me know if further information, logs, or model files are needed to diagnose this issue.
### To reproduce
1. Set up ONNX Runtime with NV TRT RTX and TensorRT execution providers.
2. Run the same ONNX model on an NVIDIA RTX 4000 Ada GPU with identical input shapes and batch size.
3. Observe and compare the inference times.
4. (Model name, input shape, and exact environment details to be filled in.)
### Urgency
_No response_
### Platform
Linux
### OS Version
sles
### ONNX Runtime Installation
Built from Source
### ONNX Runtime Version or Commit ID
a0c42367b7f74db0f529c36d7bcfc0a8c83b9f3d
### ONNX Runtime API
C++
### Architecture
X64
### Execution Provider
TensorRT, Other / Unknown
### Execution Provider Library Version
_No response_
### Model File
_No response_
### Is this a quantized model?
No
Contributor guide
Assessment
This issue has not been assessed yet.