microsoft / microsoft/onnxruntime-genai
Non-zero status code returned while running DmlFusedNode_0_0 node error running directml phi3.
- Dominant language
- C++
- Stars
- 1.1k
- Forks
- 354
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 85
Description
**Describe the bug**
When I run the code in this repository using directml, I get the following warnings and error:
2024-08-28 12:12:55.3413627 [W:onnxruntime:onnxruntime-genai, inference_session.cc:1934 onnxruntime::InferenceSession::Initialize] This model has shape massaging nodes that will execute on CPU. Use the graph capture feature with caution. As long as the intermediate shapes produced in the model using the representative input used to capture the graph, will match the shapes produced in the model for other inputs of the same shape as the representative input (common case), it is safe to use the graph capture feature.
2024-08-28 12:12:55.3512674 [W:onnxruntime:onnxruntime-genai, session_state.cc:1166 onnxruntime::VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.
2024-08-28 12:12:55.3568076 [W:onnxruntime:onnxruntime-genai, session_state.cc:1168 onnxruntime::VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.
Question: How old is Luca?
Answer:
2024-08-28 12:12:57.6011809 [E:onnxruntime:onnxruntime-genai, sequential_executor.cc:516 onnxruntime::ExecuteKernel] Non-zero status code returned while running DmlFusedNode_0_0 node. Name:'DmlFusedNode_0_0' Status Message: D:\a\_work\1\s\onnxruntime\core\providers\dml\DmlExecutionProvider\src\DmlGraphFusionHelper.cpp(353)\onnxruntime.DLL!00007FFEA707254C: (caller: 00007FFEA706B7CF) Exception(1) tid(6354) 80070057 The parameter is incorrect.
The code, instead, works perfectly when using cpu.
**To Reproduce**
Clone the following repository: https://github.com/darcome/testonnx
My laptop has an NVIDIA RTX 3080, which I don't know if it's being used, but that's another story :)
Contributor guide
No contributing guide indexed for this repository
Research direction
Clone the linked darcome/testonnx repository and reproduce the Phi-3 question using DirectML, comparing it with the working CPU run. Begin with the logged DmlFusedNode_0_0 failure and the DirectML execution path; done means the same inference completes without the parameter error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100