microsoft / microsoft/onnxruntime

[Feature Request]

Open
#21,316 2 comments 0 reactions 1 assignee Assigned to @chilo-ms View on GitHub
ep:TensorRT feature request
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the feature request

Can onnxruntime support directly loading *.engine or *.trt to initialize the session when using TensorRT EP?

### Describe scenario use case

Currently using TensorRT EP, in order to ensure the initialization speed, the *.engine is written to the hard disk when using the cache. In my production environment, I want *.engine to be encrypted. And they can be directly loaded in the following way.
//Decrypt model to byte
byte=Decryptor.read("./encrypted_model.engine");

//Initialize to session
session = Ort::Session(env, byte.data(), byte.size(), session_options);

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.