microsoft / microsoft/onnxruntime
Investigate perf implication for storing tranposed weights
Open
@justinchuby is already working on this.
Since Mar 11, 2026.
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
PyTorch stores linear weights transposed and according to chatgpt this is more efficient for the matmul computation. We cannot do this in onnx yet because the Gemm op does not support broadcasting. Create benchmark to test and show that we should update Gemm to support broadcasting and store the weights transposed.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.