microsoft / microsoft/onnxruntime-genai

[GPU Driver Error] Inference Phi-3 on Snapdragon X Elite

Open
#814 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

ep:DML
Dominant language
C++
Stars
1.1k
Forks
354
Avg merge
2d 16h
Merged PRs (30d)
85

Description

**Describe the bug**
Hello guys, Thank you for your work.
I wanna test phi-3 inference on copilot PC(Snapadragon X Elite).
I installed onnxruntime-genai-directml to use adreno gpu.

**To Reproduce**
I run the phi-3 models by referring phi-3 tutorial.
"https://github.com/microsoft/onnxruntime-genai/blob/main/examples/python/phi-3-tutorial.md"

**Screenshots**
```
(genai) c:\Users\kyung\source>python.exe phi3-qa.py -g -v -m directml\directml-int4-awq-block-128
Loading model...
Model loaded
Tokenizer created
Device Type: DirectML

Input: Generate a 200 word story of starwars.
Generator created
Running generation loop ...

Output: 2024-8019 14:57:30.3851640 [E: onnxruntime:onnxruntime-genai, sequenctial_executor.cc:516 onnxruntime::ExecuteKernel] Non-zero status code returned while running DmlFusedNode_0_0 node. Name: 'DmlFusedNode_0_0' Status Message: D:\a\_work\1\s\onnxruntime\core\providers\dml\DmlExecutionProvider\src\DmlGraphFusionHelper.cpp(576)\onnxruntime.dll!00007FFFDD326373: (caller: 00007FFFDD3B88FD) Exeption(1) tid(44dc) 887A0005 The GPU device instance has been suspended. Use GetDeviceRemovedReason to determine the appropriate action.

Traceback (most recent call last):
File "C:\Users\kyung\source\phi3-qa.py", line 105, in
main(args)
File "C:\Users\kyung\source\phi3-qa.py", line 65, in
generator.compute_logits()
onnxruntime_genai.onnxruntime_genai.OrtException: D:\a\_work\1\s\onnxruntime\core\providers\dml\DmlExecutionProvider\src\DmlCommandRecorder.cpp(371)\onnxruntime.dll!00007FFFDD33CA80: (caller: 00007FFFDD2AF694) Exception(2) tid(44dc) 887A0020 An internal issue prevented the driver form carrying out the specified operation. The driver's state is probably suspect, and the application should not continue.
```

**Desktop (please complete the following information):**
● OS: window11
● Version: onnxruntime-genai-directml(0.3.0), onnxruntime-directml(1.18.1)

**Additional context**
I got an error from GPU driver.
How to fix this issue? Have you ever tested on Qualcomm adreno GPU?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with examples/python/phi-3-tutorial.md and the reported phi3-qa.py failure at generator.compute_logits(). Compare the DirectML execution path and the versions listed in the report, focusing on the DmlFusedNode_0_0 and GPU device suspension errors. Done requires identifying a reproducible compatibility cause, fix, or supported workaround for Snapdragon X Elite/Adreno.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
ai-infra-agents
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.