microsoft / microsoft/onnxruntime
Null Tensor as output when using TensorRT without session io bindings
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the issue
When I use the TensorRT execution provider and I use session.run like this:
'''
std::vector lOutputTensors = mOrtSession->Run(Ort::RunOptions{ nullptr }, mInputNodeNamePointers.data(), lInputTensors.data(),
mInputNodeNamePointers.size(), mOutputNodeNamePointers.data(), mOutputNodeNamePointers.size());
'''
Then I get errors when calling lOutputTensors[0].GetTensorMutableData() saying that the output is null. This does not happen when I use the cuda execution provider. Also when I use session io bindings with TensorRT then I have no issues. But there are some reasons that lead to me not wanting to use session io bindings, so I would like it to work without them. I did found someone online who seemingly got it to work without session io bindings, but I saw that he was using an experimental version of onnxruntime:
https://github.com/microsoft/onnxruntime/issues/14614
So I guess a follow up question is, if it is necessary how do I build the experimental version of onnxruntime?
### To reproduce
Using TensorRT as your execution provider and then calling session.run() like this:
'''
std::vector lOutputTensors = mOrtSession->Run(Ort::RunOptions{ nullptr }, mInputNodeNamePointers.data(), lInputTensors.data(),
mInputNodeNamePointers.size(), mOutputNodeNamePointers.data(), mOutputNodeNamePointers.size());
'''
Followed by lOutputTensors[0].GetTensorMutableData()
### Urgency
_No response_
### Platform
Windows
### OS Version
10.0.19045
### ONNX Runtime Installation
Built from Source
### ONNX Runtime Version or Commit ID
1.16.1
### ONNX Runtime API
C++
### Architecture
X64
### Execution Provider
TensorRT
### Execution Provider Library Version
8.6.1.6
Contributor guide
Assessment
This issue has not been assessed yet.