microsoft / microsoft/onnxruntime
[Feature Request] Leverage Arm64 SME/SME2 instructions in onnxruntime
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the feature request
Apple M4 supports SME2 instructions that can greatly improve matrix multiplication performance, and onnxruntime may want to leverage those instructions.
This https://github.com/tzakharko/m4-sme-exploration/tree/main shows M4 SME support details and performance.
### Describe scenario use case
This should make onnxruntime run faster inference on Arm64 CPUs with SME support.
Contributor guide
Research direction
No repository files, tests, or entry points are named. Start by reviewing the linked m4-sme-exploration material and locating onnxruntime's Arm64 CPU inference and matrix multiplication paths. Done means SME/SME2 support is integrated for compatible CPUs and inference performance improves measurably.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100