microsoft / microsoft/onnxruntime
[Feature Request] ONNX model file decryption/custom I/O hooks
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the feature request
For on-premises scenarios it might be good to allow the user to provide some hooks for reading/decrypting the model file for all main EP's including the CUDA EP.
This could be added by allowing the user to specify the `.so` shared file with custom I/O functions (Something like this already exists for TRT EP) or e.g. exporting I/O functions and allowing user to override them with LD_PRELOAD'ing custom overrides
- https://github.com/microsoft/onnxruntime/issues/22496
- https://github.com/microsoft/onnxruntime/pull/17111
- https://github.com/microsoft/onnxruntime/pull/15508
Maybe also these I/O hooks could be used to implement some weight loading from S3 or from a custom user's checkpoint blob database.
### Describe scenario use case
(For on-premises scenarios it might be good to allow the user to provide some hooks for reading/decrypting the model file)
Contributor guide
Research direction
Start by reviewing the linked issue 22496 and pull requests 17111 and 15508, especially the existing TensorRT EP approach. Define the supported custom I/O hook or shared-library interface for model decryption and external storage across the main execution providers, including CUDA. Done means the scope and interface are agreed and the relevant execution providers support the requested loading scenarios.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100