Fuse quantizelinear for skip layers using multioutput fusions
Open
@shivadbhavsar is already working on this.
Since Jan 31, 2025.
Perf Improve
- Dominant language
- C++
- Stars
- 333
- Forks
- 150
- Avg merge
- 4d 19h
- Merged PRs (30d)
- 54
Description
Follow up from PR #3782
Ex resnet quantized graph after above PR:
NEW:
q -> conv -> dq -> add -> relu -> q .......... -> q -> conv -> dq -> add -> relu -> q
| |
-> step -> q -----------------------------------------> concat -> conv -> ...
Doing some experimental work, it turns out that we get a slight perf boost from moving the skip-connection quantize op before the step op, and fusing it into the previous conv-pointwise kernel. This should probably be done as 2 steps:
- 1. Write pass for multioutput fusion that can fuse to
mlir_quant_convolution_dequantizelinear_dequantizelinear_add_add_relu_quantizelinear_quantizelinear - 2. Write pass to move q before step when this fusion is possible
These should be done in this order since swapping the order of quantize op and step op is generally not preferable unless doing it for this fusion.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.