microsoft / microsoft/onnxruntime-genai

"profiling_level": "detailed" is not working for multi models qnn npu

Open
#2,037 0 comments 1 reaction 2 assignees Claimed by @baijumeswani View on GitHub
platform:mobile
Dominant language
C++
Stars
1.1k
Forks
354
Avg merge
2d 16h
Merged PRs (30d)
85

Description

**Describe the bug**
A clear and concise description of what the bug is.

For a qnn recipe, there will be multiple models in genai_config.json. But if I enable `"profiling_level": "detailed"` and `"enable_profiling"`, the second `"profiling_level": "detailed"` will not work.

```
"context_ctx": {
"filename": "context_ctx.onnx",
"session_options": {
"provider_options": [
{
"qnn": {
"htp_performance_mode": "burst",
"htp_graph_finalization_optimization_mode": "3",
"soc_model": "60",
"profiling_level": "detailed",
"profiling_file_path": "qnn1.csv"
}
}
],
"enable_profiling": "op1"
},
"run_on_token_gen": false
},
"iterator_ctx": {
"filename": "iterator_ctx.onnx",
"session_options": {
"provider_options": [
{
"qnn": {
"htp_performance_mode": "burst",
"htp_graph_finalization_optimization_mode": "3",
"soc_model": "60",
"profiling_level": "detailed",
"profiling_file_path": "qnn2.csv"
}
}
],
"enable_profiling": "op2"
},
"run_on_prompt": false
},
```

After run, op1.json op2.json and qnn1.csv exist, but no qnn2.csv.

**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

**Expected behavior**
A clear and concise description of what you expected to happen.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Desktop (please complete the following information):**
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]

**Smartphone (please complete the following information):**
- Device: [e.g. iPhone6]
- OS: [e.g. iOS8.1]
- Browser [e.g. stock browser, safari]
- Version [e.g. 22]

**Additional context**
Add any other context about the problem here.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.