microsoft / microsoft/onnxruntime-genai

Phi-3-Mini fails to execute on long prompts on Intel integrated GPU with DirectML

Open
#570 14 comments 0 reactions 1 assignee Claimed by @PatriceVignola View on GitHub
ep:DML
Dominant language
C++
Stars
1.1k
Forks
354
Avg merge
2d 16h
Merged PRs (30d)
85

Description

When I try to run short prompts (up to ~200 tokens) everything works well, however, if I increase the number of tokens in the input I get the following error:
```
Output: 2024-06-03 08:12:09.7100776 [E: onxruntime: onnxruntime-genai, sequential_executor.cc:516 onnxruntime::ExecuteKer nel] Non-zero status code returned while running DmlFusedNode_0_0 node. Name: 'DmlFusedNode_0_0' Status Message: D: \a\_wo rk \1\s \onnxruntime\core\providers\dml \DmlExecutionProvider\src\DmlGraphFusionHelper. cpp(1060)\onnxruntime.dll! 00007FFD50
С4АВ39: (caller: 00007FFD50CD96AE) Exception(2) tid(3218) 88740006 The GPU will not respond to more commands, most Likel y because of an invalid command passed by the calling application.
Traceback (most recent call last):
File "C: \Users\dungeon \onnxruntime-genai\examples\python\phi3-qa.py", Line 93, in
main(args)
File "C: \Users\dungeon \onnxruntime-genai\examples\python\phi3-qa.py", line 56, in main generator. compute_logits
onxruntime_genai.onxruntime_genai.OrtException: Non-zero status code returned while running DmlFusedNode_0_0 node. Nam
e: 'DmlFusedNode_0_0' Status Message: D: \a\_work\1\s\onnxruntime\core\providers\dml\DmlExecutionProvider\src\DmlGraphFusi
onHelper. cpp(1060) \onnxruntime.dll! 00007FFD50C4AB39: (caller: 00007FFD50CD96AE) Exception(2) tid (3218) 887A0006 The GPU will not respond to more commands, most likely because of an invalid command passed by the calling application.
```
I am running on Intel Core Ultra 155H with latest GPU driver available.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.