microsoft / microsoft/onnxruntime

[Feature Request] Restore XNNPACK Execution Provider for ONNX Runtime Web Backend

Open
#24,766 4 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

ep:Xnnpack feature request platform:web
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the feature request

We would like to request the re-introduction of the XNNPACK Execution Provider (EP) for ONNX Runtime Web.

### Motivation
We’ve benchmarked default Wasm and XNNPACK-Wasm execution providers for browser-based inference, and observed that XNNPACK EP outperforms the current WebAssembly-based EP for Mobilenetv3-like models, on various laptop devices.

Thread1
| Device | Execution Time Reduction by Switching to XNNPACK |
|:----------------:|:-------------------------------------------------:|
| TGL / i7-1165G7 | 54% |
| MTL/Ultra7 155H | 76% |
| LNL/Ultra7 268V | 79% |

In our benchmarks, XNNPACK EP achieves over 50% lower latency than the MLAS Wasm EP on all laptop devices.

We have a local build for adding xnnpack ep back on the latest onnxruntime and are willing to contribute. Would this change be welcomed?

### Describe scenario use case

Browser-based fp32 inference could benefit from the XNNPack ep.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files, tests, or implementation entry points are named. Start by reviewing the ONNX Runtime Web backend and the existing local build mentioned in the issue, then compare the default Wasm and XNNPACK-Wasm benchmark setup. Done would mean restoring the XNNPACK Execution Provider for browser-based fp32 inference with its performance benefits demonstrated.

Written by the indexing model from the issue text.

Assessment

Tech stack
wasm
Domain
machine-learning, performance, web-dev
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.