microsoft / microsoft/onnxruntime-genai

Non-zero status code returned while running DmlFusedNode_0_0 node error running directml phi3.

Open
#848 4 comments 0 reactions 0 assignees View on GitHub
ep:DML
Dominant language
C++
Stars
1.1k
Forks
354
Avg merge
2d 16h
Merged PRs (30d)
85

Description

**Describe the bug**
When I run the code in this repository using directml, I get the following warnings and error:

2024-08-28 12:12:55.3413627 [W:onnxruntime:onnxruntime-genai, inference_session.cc:1934 onnxruntime::InferenceSession::Initialize] This model has shape massaging nodes that will execute on CPU. Use the graph capture feature with caution. As long as the intermediate shapes produced in the model using the representative input used to capture the graph, will match the shapes produced in the model for other inputs of the same shape as the representative input (common case), it is safe to use the graph capture feature.
2024-08-28 12:12:55.3512674 [W:onnxruntime:onnxruntime-genai, session_state.cc:1166 onnxruntime::VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.
2024-08-28 12:12:55.3568076 [W:onnxruntime:onnxruntime-genai, session_state.cc:1168 onnxruntime::VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.
Question: How old is Luca?
Answer:
2024-08-28 12:12:57.6011809 [E:onnxruntime:onnxruntime-genai, sequential_executor.cc:516 onnxruntime::ExecuteKernel] Non-zero status code returned while running DmlFusedNode_0_0 node. Name:'DmlFusedNode_0_0' Status Message: D:\a\_work\1\s\onnxruntime\core\providers\dml\DmlExecutionProvider\src\DmlGraphFusionHelper.cpp(353)\onnxruntime.DLL!00007FFEA707254C: (caller: 00007FFEA706B7CF) Exception(1) tid(6354) 80070057 The parameter is incorrect.

The code, instead, works perfectly when using cpu.

**To Reproduce**
Clone the following repository: https://github.com/darcome/testonnx

My laptop has an NVIDIA RTX 3080, which I don't know if it's being used, but that's another story :)

Contributor guide

No contributing guide indexed for this repository

Research direction

Clone the linked darcome/testonnx repository and reproduce the Phi-3 question using DirectML, comparing it with the working CPU run. Begin with the logged DmlFusedNode_0_0 failure and the DirectML execution path; done means the same inference completes without the parameter error.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.