microsoft / microsoft/onnxruntime

[Feature Request] Override execution providers with environment variable

Open
#25,519 0 comments 0 reactions 0 assignees View on GitHub
feature request
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the feature request

Have environment variable like `ONR_PROVIDERS` that could be set to providers like `CPUExecutionProvider,DnnlExecutionProvider` that would override `get_available_providers()` and `InferenceSession("[...].onnx", providers)`

### Describe scenario use case

I'm using Python application that does
```
providers = onnxruntime.capi._pybind_state.get_available_providers()
onnxruntime.InferenceSession(model, providers=providers)
```

This causes CUDA to be used but I want to use CPU instead.
Currently it seems only way is to change application's code?

I would rather want just `ONR_PROVIDERS=CPUExecutionProvider python app.py`

Contributor guide

Open the contributing guide

Research direction

Start by tracing the Python entry points get_available_providers() and InferenceSession(..., providers), then determine where an ONR_PROVIDERS value could affect both calls. Compare the requested CPUExecutionProvider,DnnlExecutionProvider format with current provider selection. Done means the environment variable overrides the available providers and session providers without requiring application code changes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.