microsoft / microsoft/onnxruntime

[Feature Request] Expose the CoreML EP via OrtEpFactory so it appears in GetEpDevices() and participates in automatic EP selection

Open
#29,066 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

ep:CoreML ep:DML ep:QNN ep:WebGPU feature request platform:web
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 8h
Merged PRs (30d)
179

Description

Describe the feature request

Expose the CoreML Execution Provider through the plugin-EP infrastructure ('OrtEpFactory'), so that on Apple platforms it produces OrtEpDevice entries and becomes usable with the device-based APIs: GetEpDevices(), SessionOptionsAppendExecutionProvider_V2, and SetEpSelectionPolicy.

Today the internal factory registration in 'onnxruntime/core/session/plugin_ep/ep_library_internal.cc' creates factories for exactly three EPs: CPU (always), WebGPU ('USE_WEBGPU'), and DML ('USE_DML'). There is no 'CoreMLEpFactory', and CoreML is not available as a plugin library either. As a result, on macOS the device-based world consists of CPU and WebGPU only, even though the build contains CoreML and the hardware discovery layer correctly finds the Apple Neural Engine.

Observed with ORT 1.24.4 (git '2d924974ef') on macOS 26.5.1, Apple M3 Pro ('Mac15,7'):

GetAvailableProviders():
CoreMLExecutionProvider, WebGpuExecutionProvider, CPUExecutionProvider

OrtHardwareDevice list:
NPU (vendor Apple, vendor_id 0x106B) <- ANE, discovered but orphaned
GPU (vendor Apple, vendor_id 0x106B)
CPU (vendor Apple, vendor_id 0x106B)

GetEpDevices():
CPUExecutionProvider -> CPU
WebGpuExecutionProvider -> GPU
<- no CoreML entry

Consequences of the gap:
  • CoreML cannot be attached via SessionOptionsAppendExecutionProvider_V2, only the V1 string-based AppendExecutionProvider("CoreML", ...) path works.
  • Automatic selection never considers CoreML: SetEpSelectionPolicy(OrtExecutionProviderDevicePolicy_PREFER_NPU) on a Mac silently falls back to CPU/WebGPU instead of routing to the ANE through CoreML.
  • The ANE is enumerated as an NPU-type 'OrtHardwareDevice' but no EP ever claims it, so it stays orphaned in the device list.
  • Device-oriented tooling built on this API surface ('onnxruntime_perf_test --list_ep_devices', Windows ML-style selection code) does not see CoreML at all.
Describe scenario use case

Cross-platform applications that adopted the device-based selection model (the API surface promoted since ORT 1.22/1.23 and used by Windows ML) currently need a macOS-specific code branch: enumerate OrtEpDevices and use AppendExecutionProvider_V2 / selection policies on Windows (DML, plugin EPs), but fall back to the legacy V1 string API for CoreML on Apple platforms. A CoreMLEpFactory would make the same selection code work uniformly across platforms - including policies like 'PREFER_NPU' resolving to the Apple Neural Engine - and would let applications discover at runtime, through one API, which accelerator-backed EPs are actually usable on the machine.

Proposed implementation:

An Internal factory, analogous to DmlEpFactory: register a CoreMLEpFactory in 'ep_library_internal.cc' under 'USE_COREML'. Its GetSupportedDevices() could claim the NPU and GPU OrtHardwareDevices, producing OrtEpDevices whose options map onto the existing CoreML provider options - e.g. selecting the NPU device implies 'MLComputeUnits=CPUAndNeuralEngine', selecting the GPU device implies 'MLComputeUnits=CPUAndGPU', selecting the NPU + GPU devices implies 'MLComputeUnits=ALL'.

Environment:

ONNX Runtime 1.24.4 (git '2d924974ef', Release), ORT_API_VERSION 24, macOS 26.5.1, Apple M3 Pro ('Mac15,7'). Also verified against current 'main': 'ep_library_internal.cc' still registers only CPU/WebGPU/DML factories and the code base contains no CoreMLEpFactory.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in onnxruntime/core/session/plugin_ep/ep_library_internal.cc and compare the existing DmlEpFactory registration and GetSupportedDevices path. Trace CoreML’s existing provider options and hardware discovery, then verify that a CoreML factory exposes CoreML devices through GetEpDevices() and enables SessionOptionsAppendExecutionProvider_V2 and SetEpSelectionPolicy, including NPU selection.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, macos
Domain
backend-api-design, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.