onnx / onnx/optimizer

[BUG] The pass "fuse_concat_into_reshape" causes output mismatch

Open
#267 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
834
Forks
109
Avg merge
6h 55m
Merged PRs (30d)
2

Description

The pass "fuse_concat_into_reshape" causes output mismatch

Issue
Running the single pass fuse_concat_into_reshape with onnxoptimizer 0.4.2 changes numerical outputs. One real model regresses after this pass only, while the original model runs correctly with the same oracle inputs.

Environment

  • Ubuntu 20.04
  • Python 3.10
  • onnx==1.19.0
  • onnxruntime==1.19.2
  • onnxoptimizer==0.4.2 (latest)

Repro steps (run from this folder)

  1. Download and unzip the attached archive, then cd into the extracted directory

fuse_concat_into_reshape_repro.tar.gz

tar -xzvf fuse_concat_into_reshape_repro.tar.gz
cd fuse_concat_into_reshape_repro
  1. Create a Python 3.10 virtual environment and install dependencies:
    python -m venv .venv
    source .venv/bin/activate
    pip install --upgrade pip
    pip install -r requirements.txt
    
  2. Optimize the case with only fuse_concat_into_reshape (writes model.opt.onnx next to model.onnx):
    python optimize_model.py --case ./case_02937_seed43509122
    
  3. Differential test original vs optimized outputs using stored oracle inputs:
    python diff_test.py --case ./case_02937_seed43509122
    

Observed results

  • case_02937_seed43509122: overall max_abs=1.221e-04, max_rel=9.756e-04; output drift on multiple tensors (see details below).

Expected
fuse_concat_into_reshape should be semantics-preserving. Applying this pass alone should not change any output values.

Differential Test Output Details

  1. case_02937_seed43509122
Case: case_02937_seed43509122
	v17_0 (output[12]): max_abs=1.221e-04, max_rel=7.168e-04, shape=(2, 1, 16, 2)
	v49_0 (output[17]): max_abs=1.221e-04, max_rel=9.756e-04, shape=(44, 1, 16, 46, 2)
Overall: max_abs=1.221e-04, max_rel=9.756e-04

Attachments

  • README.md
  • requirements.txt
  • optimize_model.py (runs only fuse_concat_into_reshape and saves optimized model as model.opt.onnx)
  • diff_test.py (runs original and optimized model with same oracle inputs and reports their differences)
  • run_optimized_model.py (runs the optimized model to inspect outputs)
  • case_02937_seed43509122/ (original model and oracle inputs for reproduction)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with optimize_model.py and diff_test.py in the attached fuse_concat_into_reshape_repro archive, run the documented optimization and differential test, and inspect the fuse_concat_into_reshape pass behavior. Use the reported model and oracle inputs to isolate the mismatch; done means the pass preserves the original outputs for case_02937_seed43509122.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.