microsoft / microsoft/onnxruntime

[Graph Optimizer] LayerNorm fusion incorrectly matches ReduceMean with keepdims=0, causing 'Node input not found' error

Open
#30,513 1 comment 0 reactions 0 assignees View on GitHub
stale
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the issue

When loading a simple ONNX model that computes a normalized output using a sequence of operations (`ReduceMean`, `Sub`, `Pow`, `ReduceMean`, `Add`, `Sqrt`, `Div`, `Mul`), ONNX Runtime fails with an `InvalidArgument` error:

```
[ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid model. Node input 'nm' is not a graph input, initializer, or output of a previous node.
```
Related onnx issue: [https://github.com/onnx/onnx/issues/8204](https://github.com/onnx/onnx/issues/8204)

### To reproduce

```python
from onnx import helper, TensorProto
import onnxruntime as ort

eps_t = helper.make_tensor("eps_v", TensorProto.FLOAT, [], [1.0e-5])
two_t = helper.make_tensor("two_v", TensorProto.FLOAT, [], [2.0])

nodes = [
helper.make_node('Constant', inputs=[], outputs=['eps'], value=eps_t),
helper.make_node('Constant', inputs=[], outputs=['two'], value=two_t),
helper.make_node('ReduceMean', inputs=['x'], outputs=['mn'], axes=[-1], keepdims=0), # keepdims=0
helper.make_node('Sub', inputs=['x', 'mn'], outputs=['ct']),
helper.make_node('Pow', inputs=['ct', 'two'], outputs=['sq']),
helper.make_node('ReduceMean', inputs=['sq'], outputs=['vr'], axes=[-1], keepdims=0), # keepdims=0
helper.make_node('Add', inputs=['vr', 'eps'], outputs=['ve']),
helper.make_node('Sqrt', inputs=['ve'], outputs=['sd_raw']),
helper.make_node('Identity', inputs=['sd_raw'], outputs=['sd']),
helper.make_node('Div', inputs=['ct', 'sd'], outputs=['nm']),
helper.make_node('Mul', inputs=['nm', 'z'], outputs=['out']),
]

graph = helper.make_graph(nodes, 'graph_0', [
helper.make_tensor_value_info('x', TensorProto.FLOAT, [2]),
helper.make_tensor_value_info('z', TensorProto.FLOAT, [2, 1]),
], [helper.make_tensor_value_info('out', TensorProto.FLOAT, [2, 2])])

model = helper.make_model(graph, opset_imports=[helper.make_opsetid('', 11)])
sess = ort.InferenceSession(model.SerializeToString())
```

### Urgency

Not urgent. This is a bug found by fuzzing testing.

### Platform

Linux

### OS Version

Ubuntu 24.04

### ONNX Runtime Installation

Released Package

### ONNX Runtime Version or Commit ID

1.20.1

### ONNX Runtime API

Python

### Architecture

X64

### Execution Provider

Default CPU

### Execution Provider Library Version

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by running the Python InferenceSession reproducer against the graph containing ReduceMean nodes with keepdims=0. Inspect the LayerNorm fusion matching path and verify that this pattern is not treated as a valid fusion when it leaves an unresolved input. Done means the model loads without the InvalidArgument 'Node input not found' error.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.