microsoft / microsoft/onnxruntime
[Feature Request] Add Wasm Relaxed SIMD support and integer dot product instructions for ONNX Runtime Web
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
Describe the feature request
Wasm Relaxed SIMD includes integer dot product instructions, which will map to VNNI instructions on X86-64 platforms with AVX-VNNI (on ARM maybe SDOT, but I haven't tested), and can greatly improve the QGemm performance. And there may be more optimizations in the future if Relaxed SIMD is supported.
I have some local patches to add Wasm Relaxed SIMD build and VNNI dispatch for QGemmU8X8 to MLAS, and they improve Segment Anything Model performance to ~1.15x. Are such modifications welcome?
Describe scenario use case
Many Web models are quantized, and can benefit from the integer dot product instructions.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by examining MLAS and the existing ONNX Runtime Web Wasm SIMD build and dispatch paths, focusing on QGemmU8X8. Compare the reported local patches and determine the required build, dispatch, and platform coverage; done means Relaxed SIMD integer dot-product support is integrated and QGemm performance is validated across the relevant targets.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, wasm
- Domain
- machine-learning, performance, web-dev
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100