microsoft / microsoft/onnxruntime
[Feature Request] Restore XNNPACK Execution Provider for ONNX Runtime Web Backend
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the feature request
We would like to request the re-introduction of the XNNPACK Execution Provider (EP) for ONNX Runtime Web.
### Motivation
We’ve benchmarked default Wasm and XNNPACK-Wasm execution providers for browser-based inference, and observed that XNNPACK EP outperforms the current WebAssembly-based EP for Mobilenetv3-like models, on various laptop devices.
Thread1
| Device | Execution Time Reduction by Switching to XNNPACK |
|:----------------:|:-------------------------------------------------:|
| TGL / i7-1165G7 | 54% |
| MTL/Ultra7 155H | 76% |
| LNL/Ultra7 268V | 79% |
In our benchmarks, XNNPACK EP achieves over 50% lower latency than the MLAS Wasm EP on all laptop devices.
We have a local build for adding xnnpack ep back on the latest onnxruntime and are willing to contribute. Would this change be welcomed?
### Describe scenario use case
Browser-based fp32 inference could benefit from the XNNPack ep.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files, tests, or implementation entry points are named. Start by reviewing the ONNX Runtime Web backend and the existing local build mentioned in the issue, then compare the default Wasm and XNNPACK-Wasm benchmark setup. Done would mean restoring the XNNPACK Execution Provider for browser-based fp32 inference with its performance benefits demonstrated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- wasm
- Domain
- machine-learning, performance, web-dev
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100