microsoft / microsoft/onnxruntime

[Feature Request] LiteRT as unified EP for Android

Open
#29,720 2 comments 1 reaction 0 assignees View on GitHub
ep:CoreML ep:Xnnpack feature request platform:mobile
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the feature request

Would it be possible to create a "LiteRT EP" as a optimized backend for Android devices, just as the CoreML EP successfully does for iOS devices?

On iOS the ML inference can be run in a highly optimized manner by properly setting up the CoreML EP, which is awesome! Unfortunately, the state on the Android side is much trickier. Currently, the only recommended option aside from regular CPU is to use XNNPack, which is still just an optimized version of CPU and doesn't include any GPU or NPU. There's QNNPack, but that comes with setup complications and still only works on a subset of Android devices. There used to be the NNAPI option as the best choice, but since Google's deprecation it's no longer performant.

Now it seems LiteRT is the new recommended backend for doing ML inference on Android. But I don't want to switch to LiteRT, as ONNX Runtime has been a great partner in getting reliable, stable ML inference cross platforms. Hence, the ideal scenario would be if ONNX Runtime could add LiteRT as an Execution Provider in its framework, similar to how it has done that for CoreML on iOS. That way, we could happily stay with ONNX Runtime while still experiencing the best possible inference times on Android.

Thanks in advance for the consideration, and thank you for all the hard work making ONNX Runtime so awesome!

### Describe scenario use case

The use case would be to have the fastest possible inference on Android devices, through a simple unified interface.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing ONNX Runtime’s existing CoreML EP and the Android options named in the issue: XNNPack, QNNPack, and NNAPI. Determine the integration scope and LiteRT compatibility requirements; done would mean a supported LiteRT Execution Provider offering a unified, optimized Android inference path.

Written by the indexing model from the issue text.

Assessment

Tech stack
android, cpp
Domain
machine-learning, mobile-dev
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.