microsoft / microsoft/onnxruntime
[BUG] Sparse initializers causes Type Error for SparseToDenseMatMul
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the issue
I've created an ONNX model consisting solely of a SparseToDenseMatMul operator. A (data1) is a sparse initializer, and B (input1) is an input. Running the model results in the error:
> This is an invalid model. Type Error: Type 'tensor(float)' of input parameter (data1) of operator (SparseToDenseMatMul) in node (SparseToDenseMatMul1) is invalid.
It says data1 is a tensor and not a sparse_tensor. However, when I inspect the model using Netron, it shows "category: Initializer" and "layout: sparse".
### To reproduce
Attached are two models: sparse_matmul_julia.onnx created using my Julia code, and sparse_matmul.onnx created using some LLM-generated Python code below. The models are not identical but give the same error.
[models.zip](https://github.com/user-attachments/files/30855759/models.zip)
```python
import numpy as np
import onnx
from onnx import helper, TensorProto
import onnxruntime as ort
A_values = helper.make_tensor(
name="data1",
data_type=TensorProto.FLOAT,
dims=[4],
vals=[1.0, 2.0, 3.0, 4.0],
)
A_indices = helper.make_tensor(
name="data1_indices",
data_type=TensorProto.INT64,
dims=[4, 2],
vals=[
0, 0,
0, 2,
1, 1,
1, 3,
],
)
A = onnx.SparseTensorProto()
A.values.CopyFrom(A_values)
A.indices.CopyFrom(A_indices)
A.dims.extend([2, 4])
B = helper.make_tensor(
name="input1",
data_type=TensorProto.FLOAT,
dims=[4, 3],
vals=[
1.0, 2.0, 3.0,
4.0, 5.0, 6.0,
7.0, 8.0, 9.0,
10.0, 11.0, 12.0,
],
)
Y = helper.make_tensor_value_info(
"output",
TensorProto.FLOAT,
[2, 3],
)
node = helper.make_node(
"SparseToDenseMatMul",
inputs=["data1", "input1"],
outputs=["output"],
name="SparseToDenseMatMul1",
domain="com.microsoft",
)
graph = helper.make_graph(
nodes=[node],
name="SparseMatMulExample",
inputs=[],
outputs=[Y],
initializer=[B],
sparse_initializer=[A],
)
model = helper.make_model(
graph,
producer_name="python-test",
opset_imports=[
helper.make_operatorsetid("", 21),
helper.make_operatorsetid("com.microsoft", 1),
],
)
# Check the protobuf structure.
onnx.checker.check_model(model)
onnx.save(model, "sparse_matmul.onnx")
# Load and run the model with onnxruntime
sess = ort.InferenceSession("sparse_matmul.onnx", providers=["CPUExecutionProvider"])
outputs = sess.run(None, {})
print(outputs[0])
```
### Urgency
No urgency.
### Platform
Windows
### OS Version
Microsoft Windows 11 Home
### ONNX Runtime Installation
Released Package
### ONNX Runtime Version or Commit ID
1.24.4
### ONNX Runtime API
Python
### Architecture
X64
### Execution Provider
Default CPU
### Execution Provider Library Version
_No response_
Contributor guide
Research direction
Start with the attached models and the Python reproduction using ONNX Runtime 1.24.4, then trace validation and execution for the com.microsoft SparseToDenseMatMul operator with data1 supplied as a sparse_initializer. Confirm the cause of the tensor-versus-sparse_tensor type error and verify that the reproduced model loads and runs successfully afterward.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100