onnx / onnx/optimizer

[BUG] Pass “eliminate_common_subexpression“ causes output mismatch

Open
#254 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
834
Forks
109
Avg merge
6h 55m
Merged PRs (30d)
2

Description

Pass “eliminate_common_subexpression“ causes output mismatch

Issue
Running the single pass eliminate_common_subexpression with onnxoptimizer 0.3.19 changes numerical outputs. One real model regresses after this pass only, while the original model runs correctly with the same oracle inputs.

Environment

  • Ubuntu 20.04
  • Python 3.10
  • onnx==1.19.0
  • onnxruntime==1.23.2
  • onnxoptimizer==0.3.19 (latest)

Repro steps (run from this folder)

  1. Download and unzip the attached archive below, then cd into the extracted directory

eliminate_common_subexpression_repro.tar.gz

tar -xzvf eliminate_common_subexpression_repro.tar.gz
cd eliminate_common_subexpression_repro
  1. Create a Python environment (Python 3.10) and install dependencies:
python3 -m venv .venv
source .venv/bin/activate
pip install -U pip
pip install -r requirements.txt
  1. Optimize a case with only eliminate_common_subexpression:
  • python optimize_model.py --case ./case_03909_seed51206390
    (Writes model.opt.onnx next to model.onnx in the case directory.)
  1. Differential test original vs optimized outputs using stored oracle inputs:
  • python diff_test.py --case ./case_03909_seed51206390

Observed results

  • case_03909_seed51206390: overall max_abs=4.000e+00, max_rel=6.974e-04; one scalar output differs while all other outputs match bitwise.

Expected
eliminate_common_subexpression should be semantics-preserving. Applying this pass alone should not change any output values. Please investigate why the optimized graph for the above case produces different outputs and whether the pass is incorrectly rewriting shared subgraphs.

Differential Test Output Details

Case: case_03909_seed51206390
  output[0]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(62, 1, 1, 1)
  output[1]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(62, 1, 1, 1)
  output[2]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(62, 1, 1, 1)
  output[3]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(1, 2)
  output[4]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(1,)
  output[5]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(57, 62, 1, 1, 1)
  output[6]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(57, 62, 1, 1, 1)
  output[7]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(57, 62, 1, 1, 1)
  output[8]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(57, 62, 1, 1, 1)
  output[9]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(57, 62, 1, 1, 1)
  output[10]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(3534,)
  output[11]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(57, 62, 1, 1)
  output[12]: max_abs=4.000e+00, max_rel=6.974e-04, shape=()
  output[13]: max_abs=0.000e+00, max_rel=0.000e+00, shape=(57, 62, 1, 1)
Overall: max_abs=4.000e+00, max_rel=6.974e-04

Attachments

  • README.md (this document)
  • requirements.txt (dependency versions)
  • optimize_model.py (runs only eliminate_common_subexpression and saves model.opt.onnx)
  • diff_test.py (runs original vs optimized with oracle inputs and reports max_abs/max_rel)
  • case_03909_seed51206390/ (contains model.onnx which is original model, oracle.pkl which is the input data for both original and optimized model)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running optimize_model.py and diff_test.py in the supplied eliminate_common_subexpression_repro directory against case_03909_seed51206390. Compare model.onnx with model.opt.onnx and inspect the graph around the differing scalar output to determine whether shared subgraphs were rewritten incorrectly. Done means the pass preserves all outputs for the reproduction and the regression is covered by a test.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.