microsoft / microsoft/onnxruntime
[Feature Request] How to directly define a model that includes a qgemm node?
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the feature request
In this issue (https://github.com/microsoft/onnxruntime/issues/19503), we found that sometimes it's not possible to directly convert gemm to qgemm.
Moreover, on this page (https://onnx.ai/onnx/operators/onnx__MatMulInteger.html) I found that QGemm is a custom operator of onnxruntime, which means it's also not possible to directly create a QGemm node using onnx.helper.make_node.
Therefore, I would like to consult on how to directly define a model that includes a qgemm node, rather than obtaining it through conversion?
### Describe scenario use case
Sometimes it's not possible to directly convert gemm to QGemm, and I want to try QGemm instead of QLinearMatMul.
Contributor guide
Research direction
Start by reviewing the related issue 19503 and the ONNX MatMulInteger operator documentation linked in the report. Then inspect the onnx.helper.make_node entry point and ONNX Runtime's QGemm handling. Done means establishing whether a model can directly define a QGemm node and documenting the supported method or limitation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100