microsoft / microsoft/onnxruntime
bf16 kernel (OpSet13) for MatMul in CPU EP
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the issue
MatMul in ONNX OpSet 13 started to support bf16 (https://onnx.ai/onnx/operators/onnx__MatMul.html)
However, we dont see the implementation for bfloat16 in the CPU EP for matmul(13), https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/providers/cpu/math/matmul.cc#L61-L89
Is there any reason this is still not supported since OpSet was released long time ago?
If we want to implement it on our own, is there any PR i can reference to?
Ping @snnn @pranavsharma for help
### To reproduce
NA
### Urgency
_No response_
### Platform
Windows
### OS Version
11
### ONNX Runtime Installation
Built from Source
### ONNX Runtime Version or Commit ID
NA
### ONNX Runtime API
Python
### Architecture
X64
### Execution Provider
Default CPU
### Execution Provider Library Version
_No response_
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with onnxruntime/core/providers/cpu/math/matmul.cc, especially lines 61-89, and compare the implementation with the ONNX MatMul OpSet 13 specification linked in the issue. Determine the CPU EP changes needed for bfloat16 MatMul and add or update coverage so the feature is verified for the supported CPU path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100