microsoft / microsoft/onnxruntime

[BUG] Sparse initializers causes Type Error for SparseToDenseMatMul

Open
#31,730 1 comment 0 reactions 0 assignees View on GitHub
stale
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the issue

I've created an ONNX model consisting solely of a SparseToDenseMatMul operator. A (data1) is a sparse initializer, and B (input1) is an input. Running the model results in the error:

> This is an invalid model. Type Error: Type 'tensor(float)' of input parameter (data1) of operator (SparseToDenseMatMul) in node (SparseToDenseMatMul1) is invalid.

It says data1 is a tensor and not a sparse_tensor. However, when I inspect the model using Netron, it shows "category: Initializer" and "layout: sparse".

### To reproduce

Attached are two models: sparse_matmul_julia.onnx created using my Julia code, and sparse_matmul.onnx created using some LLM-generated Python code below. The models are not identical but give the same error.
[models.zip](https://github.com/user-attachments/files/30855759/models.zip)

```python
import numpy as np
import onnx
from onnx import helper, TensorProto
import onnxruntime as ort

A_values = helper.make_tensor(
name="data1",
data_type=TensorProto.FLOAT,
dims=[4],
vals=[1.0, 2.0, 3.0, 4.0],
)

A_indices = helper.make_tensor(
name="data1_indices",
data_type=TensorProto.INT64,
dims=[4, 2],
vals=[
0, 0,
0, 2,
1, 1,
1, 3,
],
)

A = onnx.SparseTensorProto()
A.values.CopyFrom(A_values)
A.indices.CopyFrom(A_indices)
A.dims.extend([2, 4])

B = helper.make_tensor(
name="input1",
data_type=TensorProto.FLOAT,
dims=[4, 3],
vals=[
1.0, 2.0, 3.0,
4.0, 5.0, 6.0,
7.0, 8.0, 9.0,
10.0, 11.0, 12.0,
],
)

Y = helper.make_tensor_value_info(
"output",
TensorProto.FLOAT,
[2, 3],
)

node = helper.make_node(
"SparseToDenseMatMul",
inputs=["data1", "input1"],
outputs=["output"],
name="SparseToDenseMatMul1",
domain="com.microsoft",
)

graph = helper.make_graph(
nodes=[node],
name="SparseMatMulExample",
inputs=[],
outputs=[Y],
initializer=[B],
sparse_initializer=[A],
)

model = helper.make_model(
graph,
producer_name="python-test",
opset_imports=[
helper.make_operatorsetid("", 21),
helper.make_operatorsetid("com.microsoft", 1),
],
)

# Check the protobuf structure.
onnx.checker.check_model(model)

onnx.save(model, "sparse_matmul.onnx")

# Load and run the model with onnxruntime
sess = ort.InferenceSession("sparse_matmul.onnx", providers=["CPUExecutionProvider"])
outputs = sess.run(None, {})
print(outputs[0])
```

### Urgency

No urgency.

### Platform

Windows

### OS Version

Microsoft Windows 11 Home

### ONNX Runtime Installation

Released Package

### ONNX Runtime Version or Commit ID

1.24.4

### ONNX Runtime API

Python

### Architecture

X64

### Execution Provider

Default CPU

### Execution Provider Library Version

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with the attached models and the Python reproduction using ONNX Runtime 1.24.4, then trace validation and execution for the com.microsoft SparseToDenseMatMul operator with data1 supplied as a sparse_initializer. Confirm the cause of the tensor-versus-sparse_tensor type error and verify that the reproduced model loads and runs successfully afterward.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.