microsoft / microsoft/onnxruntime-genai
C# Run Program on NPU (OnnxRuntime + DirectML + NPU)?
- Dominant language
- C++
- Stars
- 1.1k
- Forks
- 354
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 85
Description
Hi, big guys!
Recently I tried to run phi3 on NPU inference using DirectML as backend EP under Onnxruntime framework, but I failed!
I loaded the model and tokenizer through the Model class and Tokenizer class, but from the C# API given in the official Onnxruntime docs(https://onnxruntime.ai/docs/genai/api/csharp.html), I can't find any API about being able to set up the NPU, can someone expand on this in detail to explain how to load the model onto the NPU?(It's best if you can explain it through a demo)
Or instead do you know if there are demos on loading Phi3 or other LLM models to NPU?
Looking forward to your reply!
Below is the code I used to load the model and tokenizer using the OnnxruntimeGenai-directml library and enter the question to be answered.



Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the linked ONNX Runtime GenAI C# API documentation and the reported OnnxruntimeGenai-directml usage. Determine whether the existing API supports selecting an NPU through DirectML, then document the supported setup or provide a Phi-3/NPU demonstration; completion should make the required configuration and expected result clear.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- ai, machine-learning
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100