microsoft / microsoft/onnxruntime-genai
Phi-3-Mini fails to execute on long prompts on Intel integrated GPU with DirectML
- Dominant language
- C++
- Stars
- 1.1k
- Forks
- 354
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 85
Description
When I try to run short prompts (up to ~200 tokens) everything works well, however, if I increase the number of tokens in the input I get the following error:
```
Output: 2024-06-03 08:12:09.7100776 [E: onxruntime: onnxruntime-genai, sequential_executor.cc:516 onnxruntime::ExecuteKer nel] Non-zero status code returned while running DmlFusedNode_0_0 node. Name: 'DmlFusedNode_0_0' Status Message: D: \a\_wo rk \1\s \onnxruntime\core\providers\dml \DmlExecutionProvider\src\DmlGraphFusionHelper. cpp(1060)\onnxruntime.dll! 00007FFD50
С4АВ39: (caller: 00007FFD50CD96AE) Exception(2) tid(3218) 88740006 The GPU will not respond to more commands, most Likel y because of an invalid command passed by the calling application.
Traceback (most recent call last):
File "C: \Users\dungeon \onnxruntime-genai\examples\python\phi3-qa.py", Line 93, in
main(args)
File "C: \Users\dungeon \onnxruntime-genai\examples\python\phi3-qa.py", line 56, in main generator. compute_logits
onxruntime_genai.onxruntime_genai.OrtException: Non-zero status code returned while running DmlFusedNode_0_0 node. Nam
e: 'DmlFusedNode_0_0' Status Message: D: \a\_work\1\s\onnxruntime\core\providers\dml\DmlExecutionProvider\src\DmlGraphFusi
onHelper. cpp(1060) \onnxruntime.dll! 00007FFD50C4AB39: (caller: 00007FFD50CD96AE) Exception(2) tid (3218) 887A0006 The GPU will not respond to more commands, most likely because of an invalid command passed by the calling application.
```
I am running on Intel Core Ultra 155H with latest GPU driver available.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.