microsoft / microsoft/onnxruntime
TensorRT EP could not open shared library from ?
@oliviajain is already working on this.
Since Apr 25, 2022.
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 8h
- Merged PRs (30d)
- 179
Description
Describe the bug
When I was running yolov5 onnx modle on jetson xavier nx with onnxruntime 1.10 version, I encountererd this error.
what(): /home/robin/DataDisk/buildFromSource/onnxruntime/onnxruntime/core/providers/tensorrt/tensorrt_execution_provider.cc:554 onnxruntime::TensorrtExecutionProvider::TensorrtExecutionProvider(const onnxruntime::TensorrtExecutionProviderInfo&) [ONNXRuntimeError] : 11 : EP_FAIL : TensorRT EP could not open shared library from �
Urgency
Normal
System information
- OS Platform and Distribution (Linux Ubuntu 18.04):
- ONNX Runtime installed from (source or binary): From Source
- ONNX Runtime version: 1.10
- GCC/Compiler version (if compiling from source): onnxruntime is build by gcc7.5, my application is build by gcc11.1
- CUDA/cuDNN version: 10.2
- GPU model and memory: Jetson Xavier NX with L4t 32.6.1
To Reproduce
I checked out the onnxruntime git tag to 1.10. I don't use the latest 1.11 version because 1.11 needs tensor rt 8.2 but the tensor rt version on jetson is 8.0 at present. Apparently, the 1.11 version cannot be built successfully with the --use_tensorrt flag.
I build onnxruntime from source on jetson xavier nx, the build command is
./build.sh --config Release --update --build --parallel \
--build_shared_lib -use_tensorrt \
--cuda_home /usr/local/cuda --cudnn_home /usr/lib/aarch64-linux-gnu \
--tensorrt_home /usr/lib/aarch64-linux-gnu
After building, I copied all lib.so and include directory to a single folder. Then I linked these files manually to build my application.
set(ONNXRUNTIME_INCLUDE_DIR "${ONNX_INSTALL_DIR}/include" "${ONNX_INSTALL_DIR}/include/onnxruntime/core/session")
find_library(ONNXRUNTIME_LIBRARY NAMES onnxruntime
HINTS ${ONNX_INSTALL_DIR}
PATH_SUFFIXES lib64 lib
PATHS ${ONNX_INSTALL_DIR}
DOC "ONNXRUNTIME - Library")
find_library(ONNXRUNTIME_TENSORRT NAMES onnxruntime_providers_tensorrt
HINTS ${ONNX_INSTALL_DIR}
PATH_SUFFIXES lib64 lib
PATHS ${ONNX_INSTALL_DIR}
DOC "ONNXRUNTIME Tensorrt - Library")
target_include_directories(OnnxDetector PRIVATE ${ONNXRUNTIME_INCLUDE_DIR})
target_link_libraries(OnnxDetector PRIVATE ${ONNXRUNTIME_LIBRARY} ${ONNXRUNTIME_TENSORRT})
The application reference codes and models are comes from this repo
I tried to use tensorrt instead of normal cuda inference.
sessionOptions = Ort::SessionOptions();
std::vector<std::string> availableProviders = Ort::GetAvailableProviders();
auto trtAvailable = std::find(availableProviders.begin(), availableProviders.end(), "TensorrtExecutionProvider");
OrtTensorRTProviderOptions tensorrtOpt;
tensorrtOpt.device_id = 0;
if (device == ONNXDevices::TENSOR_RT && (trtAvailable != availableProviders.end())){
std::cout << "Inference device: Tensor rt" << std::endl;
sessionOptions.AppendExecutionProvider_TensorRT(tensorrtOpt);
}
After building, the application show the error as above. Furthermore, this application could under the normal cuda successfully.
Expected behavior
I hope the application can output inference results.
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
yolov5
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.