microsoft / microsoft/onnxruntime

[Documentation] Document TensorRT engine encryption dll specification

Open
#22,496 3 comments 0 reactions 0 assignees View on GitHub
documentation ep:TensorRT
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the documentation issue

As I understand there is now support for engine encryption but as far as I can tell there is no documentation for how to write an encryption dll. What its function's names should be, what parameters they have and their semantics.

We need this as there is no movement on the issues #22179 and #22165 which would be a better solution: Then we can keep our weights in our encrypted onnx file and provide it and the stripped engine file when creating a new Engine.

Another option which seems better than an encryption library is to provide a buffer-based API for engine caching. I guess this is somewhat problematic as the caller can't create a buffer beforehand as there is no way to know what size is required. But why not use some kind of stream API or just a simple callback like storeBytes(void*, size_t) that the EP can call when it has done its engine optimization (once or multiple times has to be defined). This way we can keep the engine data in RAM until we can encrypt it and then store it to disk. This has the advantage of not having to redistribute an extra dll which is easy to shim to get hold of the unencrypted data.

Yes now that I wrote this I feel that a dll/so to do encryption is a very bad idea. It provides the easiest possible way to get hold of the unencrypted data: Just make a tiny program that calls the function with the contents of the encrypted engine and write out the decrypted contents! Essentially no protection!

So to sum up: a) remove the dll support, or if not PLEASE DO NOT document the API to at least make it somewhat harder to abuse the dll to decrypt engines to steal their contents. b) make a in memory buffer possibility to handle engine file contents so we can encrypt it and store it as we like instead of being forced to store it in a disk file. I think this is what TensorRT itself does.

### Page / URL

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the TensorRT engine encryption DLL support described in this issue and the related issues #22179 and #22165. First determine whether the intended outcome is API documentation, removal of DLL support, or an in-memory engine-cache API. Done means the project direction is resolved and the relevant documented specification or follow-up work is identified.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
documentation, machine-learning, security
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.