microsoft / microsoft/onnxruntime

[Feature Request] ONNX model file decryption/custom I/O hooks

Open
#22,813 2 comments 0 reactions 0 assignees View on GitHub
feature request
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the feature request

For on-premises scenarios it might be good to allow the user to provide some hooks for reading/decrypting the model file for all main EP's including the CUDA EP.

This could be added by allowing the user to specify the `.so` shared file with custom I/O functions (Something like this already exists for TRT EP) or e.g. exporting I/O functions and allowing user to override them with LD_PRELOAD'ing custom overrides

- https://github.com/microsoft/onnxruntime/issues/22496
- https://github.com/microsoft/onnxruntime/pull/17111
- https://github.com/microsoft/onnxruntime/pull/15508

Maybe also these I/O hooks could be used to implement some weight loading from S3 or from a custom user's checkpoint blob database.

### Describe scenario use case

(For on-premises scenarios it might be good to allow the user to provide some hooks for reading/decrypting the model file)

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the linked issue 22496 and pull requests 17111 and 15508, especially the existing TensorRT EP approach. Define the supported custom I/O hook or shared-library interface for model decryption and external storage across the main execution providers, including CUDA. Done means the scope and interface are agreed and the relevant execution providers support the requested loading scenarios.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.