microsoft / microsoft/onnxruntime
Basic optimization can produce invalid graph for Dropout feeding folded Mul-one
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the issue
Body:
# ORT Basic optimization can produce an invalid graph for Dropout feeding folded Mul-one
## Summary
ONNX Runtime 1.23.2 and the latest tested PyPI CPU wheel 1.29.0 accept and execute this inference-mode Dropout graph at `ORT_DISABLE_ALL`, but fail from `ORT_ENABLE_BASIC` onward. After optimization, a downstream `Mul` still
references the removed Dropout output.
## Environment
- onnxruntime: 1.23.2; reproduced again on latest tested PyPI CPU wheel 1.29.0
- onnx: 1.22.0
- provider: CPUExecutionProvider
- OS: Linux x86_64
## Reproducer
Minimal graph:
```text
x -> Dropout -> mid
1.0 / 1.0 -> folded_one
mid * folded_one -> y
```
Run:
```bash
python repro.py
```
Observed:
```text
ORT_DISABLE_ALL: ok
ORT_ENABLE_BASIC: Invalid model. Node input 'mid' is not a graph input, initializer, or output of a previous node.
ORT_ENABLE_EXTENDED: same error
ORT_ENABLE_ALL: same error
```
Disabling either `EliminateDropout` or `DivMulFusion` makes `ORT_ENABLE_BASIC` work. Disabling `EliminateIdentity`, `ConstantFolding`, or `NoopElimination` does not.
## Why this looks like an optimizer interaction
The model is valid and runs when graph optimization is disabled. The inserted pattern is semantic identity in inference mode. In real-seed insertion campaigns, the same pattern failed on SqueezeNet, MobileNetV2, DeiT-tiny,
ResNet50, and ConvNeXt-tiny across two runs. The failure appears when `EliminateDropout` interacts with `DivMulFusion`, leaving a stale value reference in the graph.
Additional probe evidence:
- A producer/consumer matrix also reproduces the same failure for the commuted consumer form `Mul(folded_one, mid)`.
- The commuted pattern was inserted into a real SqueezeNet seed and failed the same way.
- Dropout variants with explicit ratio/training-mode/mask did not trigger in the probe; the failure is specific to the single-output inference Dropout elimination path feeding `DivMulFusion`.
### To reproduce
## Full minimal reproducer
Save as `repro.py` and run with `python repro.py`:
```python
#!/usr/bin/env python3
from __future__ import annotations
import argparse
import json
from pathlib import Path
import numpy as np
import onnx
import onnxruntime as ort
from onnx import TensorProto, helper, numpy_helper
def build_model(path: Path) -> None:
x = helper.make_tensor_value_info("x", TensorProto.FLOAT, [1, 3, 5, 5])
y = helper.make_tensor_value_info("y", TensorProto.FLOAT, [1, 3, 5, 5])
one_a = numpy_helper.from_array(np.asarray(1.0, dtype=np.float32), name="one_a")
one_b = numpy_helper.from_array(np.asarray(1.0, dtype=np.float32), name="one_b")
nodes = [
helper.make_node("Dropout", ["x"], ["mid"], name="producer_dropout_inference"),
helper.make_node("Div", ["one_a", "one_b"], ["folded_one"], name="producer_constant_div"),
helper.make_node("Mul", ["mid", "folded_one"], ["y"], name="consumer_mul_one"),
]
graph = helper.make_graph(nodes, "dropout_folded_mul_basic_toposort", [x], [y], initializer=[one_a, one_b])
model = helper.make_model(graph, opset_imports=[helper.make_opsetid("", 12)])
model.ir_version = 7
onnx.checker.check_model(model)
onnx.save_model(model, str(path))
def run(model_path: Path, level: str, disabled: list[str] | None = None) -> dict:
levels = {
"disable": ort.GraphOptimizationLevel.ORT_DISABLE_ALL,
"basic": ort.GraphOptimizationLevel.ORT_ENABLE_BASIC,
"extended": ort.GraphOptimizationLevel.ORT_ENABLE_EXTENDED,
"all": ort.GraphOptimizationLevel.ORT_ENABLE_ALL,
}
so = ort.SessionOptions()
so.graph_optimization_level = levels[level]
feeds = {"x": np.ones((1, 3, 5, 5), dtype=np.float32)}
kwargs = {"sess_options": so, "providers": ["CPUExecutionProvider"]}
if disabled:
kwargs["disabled_optimizers"] = disabled
try:
sess = ort.InferenceSession(str(model_path), **kwargs)
out = sess.run(None, feeds)[0]
return {"level": level, "disabled": disabled or [], "status": "ok", "sum": float(out.sum())}
except Exception as exc:
return {"level": level, "disabled": disabled or [], "status": "error", "error": f"{type(exc).__name__}: {exc}"}
def main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("--model", default="model.onnx")
parser.add_argument("--out", default="result.json")
args = parser.parse_args()
model_path = Path(args.model)
build_model(model_path)
payload = {
"onnxruntime": ort.__version__,
"providers": ort.get_available_providers(),
"runs": [run(model_path, level) for level in ("disable", "basic", "extended", "all")],
"ablation": [
run(model_path, "basic", [name])
for name in (
"EliminateDropout",
"DivMulFusion",
"Level1_RuleBasedTransformer",
"EliminateIdentity",
"NoopElimination",
"ConstantFolding",
)
],
}
Path(args.out).write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8")
print(json.dumps(payload, indent=2))
if __name__ == "__main__":
main()
```
———
### Urgency
_No response_
### Platform
Linux
### OS Version
x86_=64
### ONNX Runtime Installation
Released Package
### ONNX Runtime Version or Commit ID
1.23.2; reproduced again on latest tested PyPI CPU wheel 1.29.0
### ONNX Runtime API
Python
### Architecture
X64
### Execution Provider
Default CPU
### Execution Provider Library Version
_No response_
Contributor guide
Research direction
Start by running the provided repro.py with ORT_DISABLE_ALL and ORT_ENABLE_BASIC, then compare the EliminateDropout and DivMulFusion ablations. Trace the single-output inference Dropout elimination and folded Div/Mul pattern. Done means optimization no longer leaves a stale mid reference, and the minimal model and tested real-model patterns run successfully at basic and higher optimization levels.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100