microsoft / microsoft/onnxruntime-genai
_com_error at memory location 0x0000006A74D6D680 when calling generator.ComputeLogits()
- Dominant language
- C++
- Stars
- 1.1k
- Forks
- 354
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 85
Description
**Describe the bug**
I've tried to copy the HelloPhi example for loading and calling phi-3.5 in DirectML for my application. When I get to the [generator.ComputeLogits()](https://github.com/microsoft/onnxruntime-genai/blob/main/examples/csharp/HelloPhi/Program.cs#L101) part, I get a ton (e.g. hundreds) of the following memory errors:
```
Exception thrown at 0x00007FFFF9ED6D9A in .exe: Microsoft C++ exception: _com_error at memory location 0x0000006A74D6D680.
```
After a few minutes, the inference engine starts generating the text.
**To Reproduce**
Steps to reproduce the behavior:
1. Copy the DirectML version of the HelloPhi example as a new `Blank App, Packaged with Windows Application Packing Project (WinUI 3 in Desktop)` app with framework net8.0-windows10.0.19041.0 (n.b. I'm using WindowsSdkPackageVersion 10.0.19041.38).
2. Download and convert phi-3.5 to DirectML (or rely on the onnx dml verison at microsoft/Phi-3-mini-128k-instruct-onnx on huggingface - I've tried both models).
3. Run the app with the downloaded model (making sure to break when c++ exceptions are thrown.
4. See error
**Expected behavior**
No memory errors to show and to start generating text immediately (rather than after 3-5 minutes of the memory exceptions).
**Screenshots**



**Desktop (please complete the following information):**
- OS: Windows 11
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.