microsoft / microsoft/onnxruntime

Support advanced EP configuration (provider options, config entries) in ONNX backend API

Open
#28,209 0 comments 0 reactions 1 assignee Claimed by @titaiwangms View on GitHub
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

## Background

PR #28083 added security allowlists for `SessionOptions` and `RunOptions` kwargs in the ONNX backend API (`onnxruntime.backend`). During review, @tianleiwu noted that the backend API currently has no mechanism for advanced runtime configuration channels.

## Problem

The backend's `prepare()` function creates an `InferenceSession` but never passes `provider_options`. Similarly, `session_config_entries` and `run_config_entries` use method-based APIs (`add_session_config_entry()`, `add_run_config_entry()`) that don't map to Python properties and can't be set via the existing kwargs/setattr pattern.

This means features like CUDA graph (`enable_cuda_graph` for CUDA EP, `trt_cuda_graph_enable` for TensorRT EP) and per-run config (`gpu_graph_id`) are unreachable through the backend API.

## Proposed Solution

Add support for structured configuration parameters:

1. **Provider options**: New `provider_options` kwarg accepting a dict (e.g., `{'CUDAExecutionProvider': {'enable_cuda_graph': True}}`), with per-EP allowlists for security
2. **Session config entries**: New `session_config` dict kwarg with allowlisted safe keys
3. **Run config entries**: New `run_config` dict kwarg with allowlisted safe keys

## Scope

- Per-EP option allowlists need security auditing (some EP options can load libraries or write files)
- Config entry key space is large and EP-specific — may need prefix-based allowlisting
- Estimated ~200–400 lines of new code + tests
- Files: `onnxruntime/python/backend/backend.py`, `backend_rep.py`, test file

## Context

- PR #28083: security fix that added the allowlist pattern
- tianleiwu's review comment: https://github.com/microsoft/onnxruntime/pull/28083#pullrequestreview-4159798553

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.