microsoft / microsoft/onnxruntime
Support advanced EP configuration (provider options, config entries) in ONNX backend API
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
## Background
PR #28083 added security allowlists for `SessionOptions` and `RunOptions` kwargs in the ONNX backend API (`onnxruntime.backend`). During review, @tianleiwu noted that the backend API currently has no mechanism for advanced runtime configuration channels.
## Problem
The backend's `prepare()` function creates an `InferenceSession` but never passes `provider_options`. Similarly, `session_config_entries` and `run_config_entries` use method-based APIs (`add_session_config_entry()`, `add_run_config_entry()`) that don't map to Python properties and can't be set via the existing kwargs/setattr pattern.
This means features like CUDA graph (`enable_cuda_graph` for CUDA EP, `trt_cuda_graph_enable` for TensorRT EP) and per-run config (`gpu_graph_id`) are unreachable through the backend API.
## Proposed Solution
Add support for structured configuration parameters:
1. **Provider options**: New `provider_options` kwarg accepting a dict (e.g., `{'CUDAExecutionProvider': {'enable_cuda_graph': True}}`), with per-EP allowlists for security
2. **Session config entries**: New `session_config` dict kwarg with allowlisted safe keys
3. **Run config entries**: New `run_config` dict kwarg with allowlisted safe keys
## Scope
- Per-EP option allowlists need security auditing (some EP options can load libraries or write files)
- Config entry key space is large and EP-specific — may need prefix-based allowlisting
- Estimated ~200–400 lines of new code + tests
- Files: `onnxruntime/python/backend/backend.py`, `backend_rep.py`, test file
## Context
- PR #28083: security fix that added the allowlist pattern
- tianleiwu's review comment: https://github.com/microsoft/onnxruntime/pull/28083#pullrequestreview-4159798553
Contributor guide
Assessment
This issue has not been assessed yet.