microsoft / microsoft/onnxruntime
[Feature Request] `GetEpDevices()` returns a sorted EP devices list
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the feature request
Have [`GetEpDevices()`](https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/session/onnxruntime_c_api.h#L5177) API returning the `OrtEpDevice` list in the order that aligns with the EP device list passing to [`EpSelectionDelegate()`](https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/session/onnxruntime_c_api.h#L486). The order is defined by [`OrderDevices()`](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/session/provider_policy_context.cc#L49), whose preference is NPU -> GPU -> CPU, for multiple GPU, discrete GPU -> integrated GPU and vendor EP -> generic EP etc.
The existing `GetEpDevices()` calls [`Environment::GetOrtEpDevices()`](https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/session/environment.h#L136) internally that returns the `execution_devices_` without ordering it before. However, [`ProviderPolicyContext::SelectEpsForSession`](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/session/provider_policy_context.cc#L136) calls `EpSelectionDelegate()` after sorting the EP devices list by `OrderDevices()`.
### Describe scenario use case
Having `GetEpDevices()` to return an ordered EP devices list would allow applications, such as WebNN, to know which EP is likely to be used before creating a session. This is required to adjust the model building behavior (by calling OrtModelEditor API), for example, avoid using in-memory external weights if the likely-to-be-used EP doesn't support it, such as issue https://github.com/microsoft/onnxruntime/issues/24768, https://github.com/microsoft/onnxruntime/pull/24391 and https://github.com/microsoft/onnxruntime/issues/25304. Getting the ordered EP devices list via `EpSelectionDelegate` is too late.
Contributor guide
Research direction
Start with GetEpDevices() and Environment::GetOrtEpDevices() in the session API and environment headers, then trace ProviderPolicyContext::SelectEpsForSession and OrderDevices() in provider_policy_context.cc. Confirm the public list uses the same ordering as EpSelectionDelegate(), including the documented device preferences, and verify the affected session behavior with relevant existing tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- api, machine-learning
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 52/100