microsoft / microsoft/onnxruntime

Basic optimization can produce invalid graph for Dropout feeding folded Mul-one

Open
#32,414 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the issue

Body:

# ORT Basic optimization can produce an invalid graph for Dropout feeding folded Mul-one

## Summary

ONNX Runtime 1.23.2 and the latest tested PyPI CPU wheel 1.29.0 accept and execute this inference-mode Dropout graph at `ORT_DISABLE_ALL`, but fail from `ORT_ENABLE_BASIC` onward. After optimization, a downstream `Mul` still
references the removed Dropout output.

## Environment

- onnxruntime: 1.23.2; reproduced again on latest tested PyPI CPU wheel 1.29.0
- onnx: 1.22.0
- provider: CPUExecutionProvider
- OS: Linux x86_64

## Reproducer

Minimal graph:

```text
x -> Dropout -> mid
1.0 / 1.0 -> folded_one
mid * folded_one -> y
```

Run:

```bash
python repro.py
```

Observed:

```text
ORT_DISABLE_ALL: ok
ORT_ENABLE_BASIC: Invalid model. Node input 'mid' is not a graph input, initializer, or output of a previous node.
ORT_ENABLE_EXTENDED: same error
ORT_ENABLE_ALL: same error
```

Disabling either `EliminateDropout` or `DivMulFusion` makes `ORT_ENABLE_BASIC` work. Disabling `EliminateIdentity`, `ConstantFolding`, or `NoopElimination` does not.

## Why this looks like an optimizer interaction

The model is valid and runs when graph optimization is disabled. The inserted pattern is semantic identity in inference mode. In real-seed insertion campaigns, the same pattern failed on SqueezeNet, MobileNetV2, DeiT-tiny,
ResNet50, and ConvNeXt-tiny across two runs. The failure appears when `EliminateDropout` interacts with `DivMulFusion`, leaving a stale value reference in the graph.

Additional probe evidence:

- A producer/consumer matrix also reproduces the same failure for the commuted consumer form `Mul(folded_one, mid)`.
- The commuted pattern was inserted into a real SqueezeNet seed and failed the same way.
- Dropout variants with explicit ratio/training-mode/mask did not trigger in the probe; the failure is specific to the single-output inference Dropout elimination path feeding `DivMulFusion`.

### To reproduce

## Full minimal reproducer

Save as `repro.py` and run with `python repro.py`:

```python
#!/usr/bin/env python3
from __future__ import annotations

import argparse
import json
from pathlib import Path

import numpy as np
import onnx
import onnxruntime as ort
from onnx import TensorProto, helper, numpy_helper

def build_model(path: Path) -> None:
x = helper.make_tensor_value_info("x", TensorProto.FLOAT, [1, 3, 5, 5])
y = helper.make_tensor_value_info("y", TensorProto.FLOAT, [1, 3, 5, 5])
one_a = numpy_helper.from_array(np.asarray(1.0, dtype=np.float32), name="one_a")
one_b = numpy_helper.from_array(np.asarray(1.0, dtype=np.float32), name="one_b")
nodes = [
helper.make_node("Dropout", ["x"], ["mid"], name="producer_dropout_inference"),
helper.make_node("Div", ["one_a", "one_b"], ["folded_one"], name="producer_constant_div"),
helper.make_node("Mul", ["mid", "folded_one"], ["y"], name="consumer_mul_one"),
]
graph = helper.make_graph(nodes, "dropout_folded_mul_basic_toposort", [x], [y], initializer=[one_a, one_b])
model = helper.make_model(graph, opset_imports=[helper.make_opsetid("", 12)])
model.ir_version = 7
onnx.checker.check_model(model)
onnx.save_model(model, str(path))

def run(model_path: Path, level: str, disabled: list[str] | None = None) -> dict:
levels = {
"disable": ort.GraphOptimizationLevel.ORT_DISABLE_ALL,
"basic": ort.GraphOptimizationLevel.ORT_ENABLE_BASIC,
"extended": ort.GraphOptimizationLevel.ORT_ENABLE_EXTENDED,
"all": ort.GraphOptimizationLevel.ORT_ENABLE_ALL,
}
so = ort.SessionOptions()
so.graph_optimization_level = levels[level]
feeds = {"x": np.ones((1, 3, 5, 5), dtype=np.float32)}
kwargs = {"sess_options": so, "providers": ["CPUExecutionProvider"]}
if disabled:
kwargs["disabled_optimizers"] = disabled
try:
sess = ort.InferenceSession(str(model_path), **kwargs)
out = sess.run(None, feeds)[0]
return {"level": level, "disabled": disabled or [], "status": "ok", "sum": float(out.sum())}
except Exception as exc:
return {"level": level, "disabled": disabled or [], "status": "error", "error": f"{type(exc).__name__}: {exc}"}

def main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("--model", default="model.onnx")
parser.add_argument("--out", default="result.json")
args = parser.parse_args()
model_path = Path(args.model)
build_model(model_path)
payload = {
"onnxruntime": ort.__version__,
"providers": ort.get_available_providers(),
"runs": [run(model_path, level) for level in ("disable", "basic", "extended", "all")],
"ablation": [
run(model_path, "basic", [name])
for name in (
"EliminateDropout",
"DivMulFusion",
"Level1_RuleBasedTransformer",
"EliminateIdentity",
"NoopElimination",
"ConstantFolding",
)
],
}
Path(args.out).write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8")
print(json.dumps(payload, indent=2))

if __name__ == "__main__":
main()
```

———

### Urgency

_No response_

### Platform

Linux

### OS Version

x86_=64

### ONNX Runtime Installation

Released Package

### ONNX Runtime Version or Commit ID

1.23.2; reproduced again on latest tested PyPI CPU wheel 1.29.0

### ONNX Runtime API

Python

### Architecture

X64

### Execution Provider

Default CPU

### Execution Provider Library Version

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by running the provided repro.py with ORT_DISABLE_ALL and ORT_ENABLE_BASIC, then compare the EliminateDropout and DivMulFusion ablations. Trace the single-output inference Dropout elimination and folded Div/Mul pattern. Done means optimization no longer leaves a stale mid reference, and the minimal model and tested real-model patterns run successfully at basic and higher optimization levels.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.