microsoft / microsoft/onnxruntime-genai

Setting backend_type in QNN has no effect.

Open
#1,728 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
1.1k
Forks
354
Avg merge
2d 16h
Merged PRs (30d)
85

Description

# Describe the bug
When loading an ONNX model with QNN, setting `backend_type` has no effect.

# To Reproduce
```csharp
var config = new Microsoft.ML.OnnxRuntimeGenAI.Config(ModelDir);
config.ClearProviders();
config.AppendProvider("QNNExecutionProvider");
config.SetProviderOption("QNNExecutionProvider", "backend_type", "gpu");
config.SetProviderOption("QNNExecutionProvider", "profiling_file_path", "C:\\ProgramData\\Dynabook\\Dynabook Assistant\\Models\\test.csv");
//var path = Path.Combine(AppContext.BaseDirectory, "QnnGpu.dll");
//config.SetProviderOption("qnn", "backend_path", path);
//config.SetProviderOption("qnn", "htp_performance_mode", "high_performance");
_model = new OnnxRuntimeGenAIChatClient(config, true, options);
```

# Expected behavior
GPU execution should take effect.

# Screenshots

Image

Image

# Desktop (please complete the following information)
- OS: Windows 11 24H2 26100.4946
- Microsoft.ML.OnnxRuntimeGenAI.QNN: 0.9.0
-

Image

# Additional context
Add any other context about the problem here.

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the issue with the shown Config, SetProviderOption, and OnnxRuntimeGenAIChatClient entry points. Then trace how QNNExecutionProvider options reach the provider, focusing on backend_type and the expected GPU execution behavior. Done means setting backend_type to gpu changes execution as expected, with a regression check if the relevant test location can be identified.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, csharp
Domain
ai, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.