microsoft / microsoft/onnxruntime

Batchnorm after ConvTranspose is not being fused to ConvTranspose

Open
#14,270 4 comments 2 reactions 0 assignees View on GitHub
feature request quantization
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the issue

During the optimization stage of onnxruntime, the batchnorm preceded by Conv operator is being fused with the Conv operator and eliminated. The same process should apply to batchnorm preceded by ConvTranspose, however this is not the case as the example graph that i attach shows.
![image](https://user-images.githubusercontent.com/11621365/212178452-cb66dd83-6e93-4b9c-b27d-b9ff65c74b28.png)
and here is the output after quantization step showing the qdq onnx graph
![image](https://user-images.githubusercontent.com/11621365/212178687-b936b6ff-e1ed-4734-9249-be9ad29bb295.png)

### To reproduce

The network is shown in the above section. At this time I'm not able to upload this small onnx file to the github sever per company policies. I'm starting the process to get this material added asap

### Urgency

we are working on large networks that contain ConvTranspose and Batchnorm sections. To fully embrace the onnxruntime qdq process, it is essential to have the fusing support as well as the quantization support for ConvTranspose (a separate issue is already open https://github.com/microsoft/onnxruntime/issues/8988

### Platform

Linux

### OS Version

18.04

### ONNX Runtime Installation

Released Package

### ONNX Runtime Version or Commit ID

1.13.1

### ONNX Runtime API

Python

### Architecture

X64

### Execution Provider

Default CPU

### Execution Provider Library Version

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by obtaining the unavailable ONNX reproduction graph and observing the optimization and quantization output for the ConvTranspose/BatchNorm pattern. Compare its behavior with the existing Conv fusion behavior. Done means BatchNorm is fused with ConvTranspose and eliminated in the resulting graph, with the expected QDQ output.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.