microsoft / microsoft/onnxruntime
[Graph Optimizer] LayerNorm fusion incorrectly matches ReduceMean with keepdims=0, causing 'Node input not found' error
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the issue
When loading a simple ONNX model that computes a normalized output using a sequence of operations (`ReduceMean`, `Sub`, `Pow`, `ReduceMean`, `Add`, `Sqrt`, `Div`, `Mul`), ONNX Runtime fails with an `InvalidArgument` error:
```
[ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid model. Node input 'nm' is not a graph input, initializer, or output of a previous node.
```
Related onnx issue: [https://github.com/onnx/onnx/issues/8204](https://github.com/onnx/onnx/issues/8204)
### To reproduce
```python
from onnx import helper, TensorProto
import onnxruntime as ort
eps_t = helper.make_tensor("eps_v", TensorProto.FLOAT, [], [1.0e-5])
two_t = helper.make_tensor("two_v", TensorProto.FLOAT, [], [2.0])
nodes = [
helper.make_node('Constant', inputs=[], outputs=['eps'], value=eps_t),
helper.make_node('Constant', inputs=[], outputs=['two'], value=two_t),
helper.make_node('ReduceMean', inputs=['x'], outputs=['mn'], axes=[-1], keepdims=0), # keepdims=0
helper.make_node('Sub', inputs=['x', 'mn'], outputs=['ct']),
helper.make_node('Pow', inputs=['ct', 'two'], outputs=['sq']),
helper.make_node('ReduceMean', inputs=['sq'], outputs=['vr'], axes=[-1], keepdims=0), # keepdims=0
helper.make_node('Add', inputs=['vr', 'eps'], outputs=['ve']),
helper.make_node('Sqrt', inputs=['ve'], outputs=['sd_raw']),
helper.make_node('Identity', inputs=['sd_raw'], outputs=['sd']),
helper.make_node('Div', inputs=['ct', 'sd'], outputs=['nm']),
helper.make_node('Mul', inputs=['nm', 'z'], outputs=['out']),
]
graph = helper.make_graph(nodes, 'graph_0', [
helper.make_tensor_value_info('x', TensorProto.FLOAT, [2]),
helper.make_tensor_value_info('z', TensorProto.FLOAT, [2, 1]),
], [helper.make_tensor_value_info('out', TensorProto.FLOAT, [2, 2])])
model = helper.make_model(graph, opset_imports=[helper.make_opsetid('', 11)])
sess = ort.InferenceSession(model.SerializeToString())
```
### Urgency
Not urgent. This is a bug found by fuzzing testing.
### Platform
Linux
### OS Version
Ubuntu 24.04
### ONNX Runtime Installation
Released Package
### ONNX Runtime Version or Commit ID
1.20.1
### ONNX Runtime API
Python
### Architecture
X64
### Execution Provider
Default CPU
### Execution Provider Library Version
_No response_
Contributor guide
Research direction
Start by running the Python InferenceSession reproducer against the graph containing ReduceMean nodes with keepdims=0. Inspect the LayerNorm fusion matching path and verify that this pattern is not treated as a valid fusion when it leaves an unresolved input. Done means the model loads without the InvalidArgument 'Node input not found' error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100