microsoft / microsoft/onnxruntime-genai

BSOD when running the sample with DML runtime

Open
#1,007 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

crash ep:DML
Dominant language
C++
Stars
1.1k
Forks
354
Avg merge
2d 16h
Merged PRs (30d)
85

Description

**Describe the bug**
When running one of the examples for directml OnmxRuntimeGenAI, I get the blue screen of death on my Surface Lpatop Studio 2

**To Reproduce**
Steps to reproduce the behavior:
1. Use this sample (any would do):
```
using Microsoft.ML.OnnxRuntimeGenAI;
using System.Reflection.Emit;
using System.Reflection;

var modelPath = @"C:\Models\microsoft\Phi-3-medium-4k-instruct-onnx-directml\directml-int4-awq-block-128";
var model = new Model(modelPath);
var tokenizer = new Tokenizer(model);

var systemPrompt = "You are an AI assistant that helps people find information. Answer questions using a direct style. Do not share more information that the requested by the users.";

// chat start
Console.WriteLine(@"Ask your question. Type an empty string to Exit.");

// chat loop
while (true)
{
// Get user question
Console.WriteLine();
Console.Write(@"Q: ");
var userQ = Console.ReadLine();
if (string.IsNullOrEmpty(userQ))
{
break;
}

// show phi3 response
Console.Write("Phi3: ");
var fullPrompt = $"<|system|>{systemPrompt}<|end|><|user|>{userQ}<|end|><|assistant|>";
var tokens = tokenizer.Encode(fullPrompt);

var generatorParams = new GeneratorParams(model);
generatorParams.SetSearchOption("max_length", 2048);
generatorParams.SetSearchOption("past_present_share_buffer", false);
generatorParams.SetInputSequences(tokens);

var generator = new Generator(model, generatorParams);
while (!generator.IsDone())
{
generator.ComputeLogits();
generator.GenerateNextToken();
var outputTokens = generator.GetSequence(0);
var newToken = outputTokens.Slice(outputTokens.Length - 1, 1);
var output = tokenizer.Decode(newToken);
Console.Write(output);
}
Console.WriteLine();
}
```

2. Have this Csproj:
```


Exe
net8.0
enable
enable
x64
win-x64




```
3. Run the sample with the model from here: https://huggingface.co/microsoft/Phi-3-medium-4k-instruct-onnx-directml/tree/main
4. Say hi

**Expected behavior**
Getting a greeting or response from the model (preferably using the NPU or at least the GPU)

**Screenshots**
Can share a video recorded using my phone with the BSOD. (contact me on Teams if needed).

**Desktop (please complete the following information):**
- Device: Surface Laptop Studio 2
- OS: Windows 11 Enterprise 24H2
- Browser: n/a
- Version: 0.4.0 (C# `Microsoft.ML.OnnxRuntimeGenAI.DirectML` and `Microsoft.ML.OnnxRuntimeGenAI`)
- Intel NPU Driver: 31.0.100.2016 (latest available for the NPU)
- Nvidia GeForce RTX 4060 Driver: 31.0.15.3878

**Additional context**
- CPU Runtime works as expected
- Will test CUDA Runtime as well and report the result in the comment.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the BSOD with the provided C# sample, net8.0 x64 csproj, Phi-3 DirectML model, and the listed Microsoft.ML.OnnxRuntimeGenAI.DirectML 0.4.0 package on the reported Surface Laptop Studio 2. Compare the DirectML run with the CPU runtime and document a configuration or runtime change that prevents the crash while producing a model response.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
machine-learning, operating-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.