microsoft / microsoft/onnxruntime

[Feature Request] How to directly define a model that includes a qgemm node?

Open
#22,743 0 comments 0 reactions 0 assignees View on GitHub
feature request
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the feature request

In this issue (https://github.com/microsoft/onnxruntime/issues/19503), we found that sometimes it's not possible to directly convert gemm to qgemm.

Moreover, on this page (https://onnx.ai/onnx/operators/onnx__MatMulInteger.html) I found that QGemm is a custom operator of onnxruntime, which means it's also not possible to directly create a QGemm node using onnx.helper.make_node.

Therefore, I would like to consult on how to directly define a model that includes a qgemm node, rather than obtaining it through conversion?

### Describe scenario use case

Sometimes it's not possible to directly convert gemm to QGemm, and I want to try QGemm instead of QLinearMatMul.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the related issue 19503 and the ONNX MatMulInteger operator documentation linked in the report. Then inspect the onnx.helper.make_node entry point and ONNX Runtime's QGemm handling. Done means establishing whether a model can directly define a QGemm node and documenting the supported method or limitation.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.