microsoft / microsoft/onnxruntime-genai
BSOD when running the sample with DML runtime
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1.1k
- Forks
- 354
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 85
Description
**Describe the bug**
When running one of the examples for directml OnmxRuntimeGenAI, I get the blue screen of death on my Surface Lpatop Studio 2
**To Reproduce**
Steps to reproduce the behavior:
1. Use this sample (any would do):
```
using Microsoft.ML.OnnxRuntimeGenAI;
using System.Reflection.Emit;
using System.Reflection;
var modelPath = @"C:\Models\microsoft\Phi-3-medium-4k-instruct-onnx-directml\directml-int4-awq-block-128";
var model = new Model(modelPath);
var tokenizer = new Tokenizer(model);
var systemPrompt = "You are an AI assistant that helps people find information. Answer questions using a direct style. Do not share more information that the requested by the users.";
// chat start
Console.WriteLine(@"Ask your question. Type an empty string to Exit.");
// chat loop
while (true)
{
// Get user question
Console.WriteLine();
Console.Write(@"Q: ");
var userQ = Console.ReadLine();
if (string.IsNullOrEmpty(userQ))
{
break;
}
// show phi3 response
Console.Write("Phi3: ");
var fullPrompt = $"<|system|>{systemPrompt}<|end|><|user|>{userQ}<|end|><|assistant|>";
var tokens = tokenizer.Encode(fullPrompt);
var generatorParams = new GeneratorParams(model);
generatorParams.SetSearchOption("max_length", 2048);
generatorParams.SetSearchOption("past_present_share_buffer", false);
generatorParams.SetInputSequences(tokens);
var generator = new Generator(model, generatorParams);
while (!generator.IsDone())
{
generator.ComputeLogits();
generator.GenerateNextToken();
var outputTokens = generator.GetSequence(0);
var newToken = outputTokens.Slice(outputTokens.Length - 1, 1);
var output = tokenizer.Decode(newToken);
Console.Write(output);
}
Console.WriteLine();
}
```
2. Have this Csproj:
```
Exe
net8.0
enable
enable
x64
win-x64
```
3. Run the sample with the model from here: https://huggingface.co/microsoft/Phi-3-medium-4k-instruct-onnx-directml/tree/main
4. Say hi
**Expected behavior**
Getting a greeting or response from the model (preferably using the NPU or at least the GPU)
**Screenshots**
Can share a video recorded using my phone with the BSOD. (contact me on Teams if needed).
**Desktop (please complete the following information):**
- Device: Surface Laptop Studio 2
- OS: Windows 11 Enterprise 24H2
- Browser: n/a
- Version: 0.4.0 (C# `Microsoft.ML.OnnxRuntimeGenAI.DirectML` and `Microsoft.ML.OnnxRuntimeGenAI`)
- Intel NPU Driver: 31.0.100.2016 (latest available for the NPU)
- Nvidia GeForce RTX 4060 Driver: 31.0.15.3878
**Additional context**
- CPU Runtime works as expected
- Will test CUDA Runtime as well and report the result in the comment.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the BSOD with the provided C# sample, net8.0 x64 csproj, Phi-3 DirectML model, and the listed Microsoft.ML.OnnxRuntimeGenAI.DirectML 0.4.0 package on the reported Surface Laptop Studio 2. Compare the DirectML run with the CPU runtime and document a configuration or runtime change that prevents the crash while producing a model response.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- machine-learning, operating-systems
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100