microsoft / microsoft/onnxruntime

[Feature Request] `GetEpDevices()` returns a sorted EP devices list

Open
#25,302 3 comments 3 reactions 0 assignees View on GitHub
feature request WebNN
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the feature request

Have [`GetEpDevices()`](https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/session/onnxruntime_c_api.h#L5177) API returning the `OrtEpDevice` list in the order that aligns with the EP device list passing to [`EpSelectionDelegate()`](https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/session/onnxruntime_c_api.h#L486). The order is defined by [`OrderDevices()`](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/session/provider_policy_context.cc#L49), whose preference is NPU -> GPU -> CPU, for multiple GPU, discrete GPU -> integrated GPU and vendor EP -> generic EP etc.

The existing `GetEpDevices()` calls [`Environment::GetOrtEpDevices()`](https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/session/environment.h#L136) internally that returns the `execution_devices_` without ordering it before. However, [`ProviderPolicyContext::SelectEpsForSession`](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/session/provider_policy_context.cc#L136) calls `EpSelectionDelegate()` after sorting the EP devices list by `OrderDevices()`.

### Describe scenario use case

Having `GetEpDevices()` to return an ordered EP devices list would allow applications, such as WebNN, to know which EP is likely to be used before creating a session. This is required to adjust the model building behavior (by calling OrtModelEditor API), for example, avoid using in-memory external weights if the likely-to-be-used EP doesn't support it, such as issue https://github.com/microsoft/onnxruntime/issues/24768, https://github.com/microsoft/onnxruntime/pull/24391 and https://github.com/microsoft/onnxruntime/issues/25304. Getting the ordered EP devices list via `EpSelectionDelegate` is too late.

Contributor guide

Open the contributing guide

Research direction

Start with GetEpDevices() and Environment::GetOrtEpDevices() in the session API and environment headers, then trace ProviderPolicyContext::SelectEpsForSession and OrderDevices() in provider_policy_context.cc. Confirm the public list uses the same ordering as EpSelectionDelegate(), including the documented device preferences, and verify the affected session behavior with relevant existing tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
api, machine-learning
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.