microsoft / microsoft/onnxruntime

C# I need to run the program on NPU (OnnxRuntime + DirectML + NPU),but it failed

Open
#19,846 26 comments 1 reaction 0 assignees View on GitHub
ep:DML platform:windows
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the issue

Hello, big guys, I checked the Microsoft blog and did the following test
https://blogs.windows.com/windowsdeveloper/2024/02/01/introducing-neural-processor-unit-npu-support-in-directml-developer-preview/

![image](https://github.com/microsoft/onnxruntime/assets/162970157/b8c72097-eee0-47e7-bda6-0a1e5cc60b87)

I use Windows 11 devices with Intel® Core™ Ultra processors with Intel® AI boost.
I have the latest NPU driver installed on my device, and intel openvino can reason successfully based on NPU

C# I need to run the program on NPU (OnnxRuntime + DirectML + NPU),but it failed。
so, Install Microsoft.AI.DirectML 1.13.1 and the Microsoft.ML.OnnxRuntime.DirectML 1.17.1 on VS2022 Nuget
![image](https://github.com/microsoft/onnxruntime/assets/162970157/3d115b4e-4fae-4ddb-b9d4-de5007a1d481)

Write C# test code
SessionOptions options = new SessionOptions();
options.AppendExecutionProvider_DMl(0) // GPU

Call AppendExecutionProvider_DMl function and set the parameter 0 (GPU). Inference can be successful based on GPU.

SessionOptions options = new SessionOptions();
options.AppendExecutionProvider_DMl(1) // NPU
ET but set parameter 1 (NPU), inference based on NPU error, error is as follows:
Microsoft.ML.OnnxRuntime.OnnxRuntimeException: '[ErrorCode:RuntimeException] D:\a\_work\1\s\onnxruntime\core\providers\dml\dml_provider_factory.cc(442)\onnxruntime.DLL!00007FF8DEFC3304: (caller: 00007FF8DEFC35B8) Exception(1) tid(5da8) C0262002 Specified display adapter handle is invalid.
![image](https://github.com/microsoft/onnxruntime/assets/162970157/4b2d421c-92f2-440d-93c3-f16e932c027d)

Query the source code of OnnxRuntime github, but build a C++ unit test environment to test some functions in the dml_provider_factory.cc file, the test code can be run on the device to enumerate the NPU device adapter information, and then 442 lines of code does not throw an exception.

Question 1: Why does C# code get an error by calling the NuGet package onnxruntime+directml+npu?
Question 2: How does onnxruntime+directml run on npu?

thanks

### To reproduce

// c# code
static void Main(string[] args)
{
var availableProviders = OrtEnv.Instance().GetAvailableProviders();

foreach (var provider in availableProviders) {
Console.WriteLine(provider);
}

SessionOptions options = new SessionOptions();
options.AppendExecutionProvider_DML(1)
}

// c++ code

[dml_provider_test.zip](https://github.com/microsoft/onnxruntime/files/14556389/dml_provider_test.zip)

### Urgency

_No response_

### Platform

Windows

### OS Version

Windows 11 Pro 22H2 22621.3155

### ONNX Runtime Installation

Built from Source

### ONNX Runtime Version or Commit ID

1.17.1

### ONNX Runtime API

C#

### Architecture

X64

### Execution Provider

DirectML

### Execution Provider Library Version

Microsoft.AI.DirectML 1.13.1

Contributor guide

Open the contributing guide

Research direction

Start with the C# reproduction using SessionOptions.AppendExecutionProvider_DML(1), the reported DirectML exception, and core/providers/dml/dml_provider_factory.cc around line 442. Compare the C# NuGet path with the attached dml_provider_test.zip and the working GPU and C++ cases; done means identifying and documenting or fixing the NPU adapter failure for ONNX Runtime 1.17.1.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.