NVIDIA / NVIDIA/TensorRT

Engine cache and model security

Open
#4,290 2 comments 0 reactions 1 assignee View on GitHub

@yuanyao-nv is already working on this.

Since Dec 23, 2024.

Module:ONNX triaged
Dominant language
C++
Stars
13.4k
Forks
2.4k
Avg merge
5d 3h
Merged PRs (30d)
2

Description

I am using ONNX Runtime to generate a TensorRt engine from ONNX model file.
I am using trt_options.trt_engine_cache_enable = 1; and trt_options.trt_engine_cache_path = "./path"; to generate the engine file once and to load it on later runs, which works well (speeds up strongly).

However I'm not sure about what exactly that engines file is.
Does it include models weights?
Can it be used to load the full model, without having access to the .onnx model file?

My question arises because we are encrypting our model files and load them from RAM during runtime, so that other people with access to the system don't have access to our models. If the cache is basically a full access to the model, we will need a different solution.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.