[BUG] The pass "eliminate_shape_op" causes output mismatch
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 834
- Forks
- 109
- Avg merge
- 6h 55m
- Merged PRs (30d)
- 2
Description
The pass "eliminate_shape_op" causes output mismatch
Issue
eliminate_shape_op (onnxoptimizer 0.4.2) causes output drift in a subset of filtered non-ok cases even when ORT graph optimizations are disabled. case_03181_seed45441358 shows mismatches (v4_0, v25_0), and case_05898_seed66957281 shows NaN mismatches (v14_0).
Environment
- Ubuntu 20.04
- Python 3.10
- onnx==1.19.0
- onnxruntime==1.19.2
- onnxoptimizer==0.4.2 (latest)
Repro steps (run from this folder)
- Download and unzip the attached archive below, then
cdinto the extracted directory
eliminate_shape_op_repro.tar.gz
tar -xzvf eliminate_shape_op_repro.tar.gz
cd eliminate_shape_op_repro
- Create a Python 3.10 virtual environment and install dependencies:
python -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt
- Optimize each case with only
eliminate_shape_op:
python optimize_model.py --case case_03181_seed45441358
- Run the differential test (ORT optimizations disabled) to compare optimized vs reference:
python diff_test.py --case case_03181_seed45441358
Expected
eliminate_shape_op should be semantics-preserving; applying the pass alone should not change any output values.
Differential Test Output Details
- case_03181_seed45441358
Case: case_03181_seed45441358
Mismatched outputs (optimized vs reference):
v4_0 (output[11]): max_abs=3.352e+00, max_rel=6.005e-01, shape=(1, 135, 17)
v25_0 (output[12]): max_abs=5.664e+00, max_rel=4.489e+00, shape=(57, 45, 17)
Overall: max_abs=5.664e+00, max_rel=4.489e+00
Attachments
README.md(this document)requirements.txt(dependency versions)optimize_model.py(runs onlyfuse_consecutive_squeezesand savesmodel.opt.onnx)diff_test.py(runs original vs optimized with oracle inputs and reports max_abs/max_rel)case_03181_seed45441358/(containsmodel.onnxoriginal model,oracle.pklinput data for both original and optimized models)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running optimize_model.py for case_03181_seed45441358, then use diff_test.py with ORT graph optimizations disabled to reproduce the mismatch. Read the pass-related behavior described in the repro and compare the optimized model with model.onnx using oracle.pkl. Done means applying eliminate_shape_op alone preserves all output values, including the reported cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- machine-learning, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100