microsoft / microsoft/onnxruntime

[Performance] QNN EP Leaving QDQ Nodes in the QNN Graph

Open
#20,533 0 comments 0 reactions 1 assignee Claimed by @adrianlizarraga View on GitHub
ep:QNN performance quantization
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the issue

The QNN EP appears to be leaving QDQ nodes in the QNN graph for Mul and Sub, rather than inserting quantized ops. I have included a sample model below.

### To reproduce

See attached model.

Affects Inception_v3 among other models.

### Urgency

_No response_

### Platform

Linux

### OS Version

Ubuntu 22.04

### ONNX Runtime Installation

Built from Source

### ONNX Runtime Version or Commit ID

Latest

### ONNX Runtime API

Python

### Architecture

ARM64

### Execution Provider

QNN EP

### Execution Provider Library Version

QNN 2.20

### Model File

[qdq_min_mul_sub_example.zip](https://github.com/microsoft/onnxruntime/files/15179000/qdq_min_mul_sub_example.zip)

### Is this a quantized model?

Yes

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.