microsoft / microsoft/onnxruntime-genai
[GPU Driver Error] Inference Phi-3 on Snapdragon X Elite
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1.1k
- Forks
- 354
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 85
Description
**Describe the bug**
Hello guys, Thank you for your work.
I wanna test phi-3 inference on copilot PC(Snapadragon X Elite).
I installed onnxruntime-genai-directml to use adreno gpu.
**To Reproduce**
I run the phi-3 models by referring phi-3 tutorial.
"https://github.com/microsoft/onnxruntime-genai/blob/main/examples/python/phi-3-tutorial.md"
**Screenshots**
```
(genai) c:\Users\kyung\source>python.exe phi3-qa.py -g -v -m directml\directml-int4-awq-block-128
Loading model...
Model loaded
Tokenizer created
Device Type: DirectML
Input: Generate a 200 word story of starwars.
Generator created
Running generation loop ...
Output: 2024-8019 14:57:30.3851640 [E: onnxruntime:onnxruntime-genai, sequenctial_executor.cc:516 onnxruntime::ExecuteKernel] Non-zero status code returned while running DmlFusedNode_0_0 node. Name: 'DmlFusedNode_0_0' Status Message: D:\a\_work\1\s\onnxruntime\core\providers\dml\DmlExecutionProvider\src\DmlGraphFusionHelper.cpp(576)\onnxruntime.dll!00007FFFDD326373: (caller: 00007FFFDD3B88FD) Exeption(1) tid(44dc) 887A0005 The GPU device instance has been suspended. Use GetDeviceRemovedReason to determine the appropriate action.
Traceback (most recent call last):
File "C:\Users\kyung\source\phi3-qa.py", line 105, in
main(args)
File "C:\Users\kyung\source\phi3-qa.py", line 65, in
generator.compute_logits()
onnxruntime_genai.onnxruntime_genai.OrtException: D:\a\_work\1\s\onnxruntime\core\providers\dml\DmlExecutionProvider\src\DmlCommandRecorder.cpp(371)\onnxruntime.dll!00007FFFDD33CA80: (caller: 00007FFFDD2AF694) Exception(2) tid(44dc) 887A0020 An internal issue prevented the driver form carrying out the specified operation. The driver's state is probably suspect, and the application should not continue.
```
**Desktop (please complete the following information):**
● OS: window11
● Version: onnxruntime-genai-directml(0.3.0), onnxruntime-directml(1.18.1)
**Additional context**
I got an error from GPU driver.
How to fix this issue? Have you ever tested on Qualcomm adreno GPU?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with examples/python/phi-3-tutorial.md and the reported phi3-qa.py failure at generator.compute_logits(). Compare the DirectML execution path and the versions listed in the report, focusing on the DmlFusedNode_0_0 and GPU device suspension errors. Done requires identifying a reproducible compatibility cause, fix, or supported workaround for Snapdragon X Elite/Adreno.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- ai-infra-agents
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100