microsoft / microsoft/onnxruntime

[Feature Request] Support QNN HTA backend through better Op validation

Open
#26,739 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

ep:QNN feature request
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

Describe the feature request

Currently, the QNN Execution Provider, only exposes the cpu, gpu, htp, and saver backends through the backend_type config option. While other backends are supported through the backend_path option (e.g: Qnn DSP backend), I discovered during testing that the Qnn HTA backend does not seem to work, even when enabling the CPU EP fallback (testing done using onnxruntime 1.24.0, and qnn_sdk 2.38.0.250901 / 2.41.0.251128).

Digging a bit closer, the issue seemed to be caused by bad operation validation in the qnn BaseOpBuilder. By default, onnxruntime automatically inserts transpose nodes to compensate for the difference in preferred data layout (NCHW vs NHWC). The issue lies in that, currently, transpose nodes are unsupported for the HTA backend. Normally, this wouldn't be an issue if the CPU EP fallback is enabled, as the unsupported nodes would just fallback to the CPU EP. Sadly, the IsOpSupported validation incorrectly marks the unsupported operations as supported, so that the fallback does not correctly take place

Simply fixing the IsOpSupported check to correctly mark unsupported operations as unsupported resolves the issue, allowing users to use the HTA backend

The feature request here is, therefore, to:

  • Add better operation validation for the QNN EP (e.g: the QNN sdk offers a backendGetSupportedOperations, which can be used to verify if the current operation is supported on the loaded backend)
  • Similarly to the htp backend, expose the dsp and hta backends through the backend_type config option
Describe scenario use case

Anywhere where the user would like to use the QNN HTA backend (especially relevant for older chipsets where the hardware required for the HTP backend is not available)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the QNN execution provider's BaseOpBuilder and the IsOpSupported validation described in the issue. Check how backend_type currently handles cpu, gpu, htp, and saver, then verify the QNN SDK's backendGetSupportedOperations behavior. Done means unsupported HTA operations fall back correctly and dsp and hta can be selected through backend_type.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend-api-design, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.