microsoft / microsoft/onnxruntime
[Feature Request]
Open
ep:TensorRT
feature request
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the feature request
Can onnxruntime support directly loading *.engine or *.trt to initialize the session when using TensorRT EP?
### Describe scenario use case
Currently using TensorRT EP, in order to ensure the initialization speed, the *.engine is written to the hard disk when using the cache. In my production environment, I want *.engine to be encrypted. And they can be directly loaded in the following way.
//Decrypt model to byte
byte=Decryptor.read("./encrypted_model.engine");
//Initialize to session
session = Ort::Session(env, byte.data(), byte.size(), session_options);
Contributor guide
Assessment
This issue has not been assessed yet.