microsoft / microsoft/onnxruntime-genai
"profiling_level": "detailed" is not working for multi models qnn npu
- Dominant language
- C++
- Stars
- 1.1k
- Forks
- 354
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 85
Description
**Describe the bug**
A clear and concise description of what the bug is.
For a qnn recipe, there will be multiple models in genai_config.json. But if I enable `"profiling_level": "detailed"` and `"enable_profiling"`, the second `"profiling_level": "detailed"` will not work.
```
"context_ctx": {
"filename": "context_ctx.onnx",
"session_options": {
"provider_options": [
{
"qnn": {
"htp_performance_mode": "burst",
"htp_graph_finalization_optimization_mode": "3",
"soc_model": "60",
"profiling_level": "detailed",
"profiling_file_path": "qnn1.csv"
}
}
],
"enable_profiling": "op1"
},
"run_on_token_gen": false
},
"iterator_ctx": {
"filename": "iterator_ctx.onnx",
"session_options": {
"provider_options": [
{
"qnn": {
"htp_performance_mode": "burst",
"htp_graph_finalization_optimization_mode": "3",
"soc_model": "60",
"profiling_level": "detailed",
"profiling_file_path": "qnn2.csv"
}
}
],
"enable_profiling": "op2"
},
"run_on_prompt": false
},
```
After run, op1.json op2.json and qnn1.csv exist, but no qnn2.csv.
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Desktop (please complete the following information):**
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
**Smartphone (please complete the following information):**
- Device: [e.g. iPhone6]
- OS: [e.g. iOS8.1]
- Browser [e.g. stock browser, safari]
- Version [e.g. 22]
**Additional context**
Add any other context about the problem here.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.