microsoft / microsoft/onnxruntime-genai

Crash when loading a model the 2nd time after recreating OgaHandle

Open
#1,896 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
1.1k
Forks
354
Avg merge
2d 16h
Merged PRs (30d)
85

Description

## Describe the bug
Creating an `OgaHandle`, loading a model, deleting both of them, and creating a new `OgaHandle` causes a crash. This happens in both C# and C++.

## To Reproduce
Steps to reproduce the behavior:
1. Create an `OgaHandle`
2. Load any model (I'm using [this one](https://github.com/microsoft/onnxruntime-genai/tree/main/test/test_models/hf-internal-testing/tiny-random-gpt2-fp32) with a chat template manually added to `tokenizer_config.json`)
3. Destroy or dispose the model
4. Destroy or dispose the `OgaHandle`
5. Create a new `OgaHandle`
6. Load any model again

The crash doesn't happen if a model isn't loaded before destroying the `OgaHandle`. It also happens with the Deepseek Qwen distills and it also doesn't matter if the model was used for generation.

You may use the code in [this repository](https://github.com/Zagusan/OgaHandle-Repro).

## Expected behavior
Since `OgaHandle` is essential for cleaning up resources, it should allow for loading another model after destroying and recreating it

## Environment
- OS: Windows 11 23H2
- Device: CPU
- Tested OGA versions: 0.11.2, 0.11.3, 0.11.4, and 0.12.2 (downloaded through NuGet)
- Tested ONNX Runtime version: 1.23.2 and 1.24.1

## Additional context
### C# error:
```
Exception thrown at 0x00007FFFC52F593C (onnxruntime-genai.dll) in OgaHandleRepro.exe: 0xC0000005: Access violation reading location 0x0000000000000000.
```
### C# stack trace:
```
onnxruntime-genai.dll!00007fff9dcb593c() Unknown
onnxruntime-genai.dll!00007fff9dd10d2b() Unknown
[Managed to Native Transition]
Microsoft.ML.OnnxRuntimeGenAI.dll!Microsoft.ML.OnnxRuntimeGenAI.Model.Model(Microsoft.ML.OnnxRuntimeGenAI.Config config) Unknown
OgaHandleRepro.dll!Repro.Program.LoadRandomGPT2() Line 33 C#
OgaHandleRepro.dll!Repro.Program.Main(string[] args) Line 55 C#
[Native to Managed Transition]
hostpolicy.dll!00007fffa015da9a() Unknown
hostpolicy.dll!00007fffa015dd3c() Unknown
hostpolicy.dll!00007fffa015ecb1() Unknown
hostfxr.dll!00007fffa449d56b() Unknown
hostfxr.dll!00007fffa44a029c() Unknown
hostfxr.dll!00007fffa44a2676() Unknown
hostfxr.dll!00007fffa44a079d() Unknown
hostfxr.dll!00007fffa4498998() Unknown
OgaHandleRepro.exe!00007ff7c7b12dec() Unknown
OgaHandleRepro.exe!00007ff7c7b13256() Unknown
OgaHandleRepro.exe!00007ff7c7b22548() Unknown
kernel32.dll!00007ff89380259d() Unknown
ntdll.dll!00007ff894a8af78() Unknown
```
### C++ error:
```
Unhandled exception at 0x00007FF89204055C in OgaHandleRepro.exe: Microsoft C++ exception: std::runtime_error at memory location 0x00000068F391F4D8.
```
The C++ error changed slightly in version 0.11.3 and beyond. Now, both C# and C++have the same access violation error:
```
Exception thrown at 0x00007FFB86A9720C (onnxruntime-genai.dll) in OgaHandleRepro.exe: 0xC0000005: Access violation reading location 0x0000000000000000.
```
### C++ stack trace:
```
onnxruntime-genai.dll!00007fffaad4593c() Unknown
onnxruntime-genai.dll!00007fffaada0d2b() Unknown
OgaHandleRepro.exe!OgaModel::Create(const OgaConfig & config) Line 240 C++
OgaHandleRepro.exe!LoadRandomGPT2() Line 33 C++
OgaHandleRepro.exe!main() Line 54 C++
OgaHandleRepro.exe!invoke_main() Line 79 C++
OgaHandleRepro.exe!__scrt_common_main_seh() Line 288 C++
OgaHandleRepro.exe!__scrt_common_main() Line 331 C++
OgaHandleRepro.exe!mainCRTStartup(void * __formal) Line 17 C++
kernel32.dll!00007ff89380259d() Unknown
ntdll.dll!00007ff894a8af78() Unknown
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the lifecycle in the linked OgaHandle-Repro repository, then inspect OgaModel::Create at line 240 and the C# Model.Model constructor shown in the stack traces. Compare the first and second OgaHandle/model loads in both C++ and C#. Done means recreating the handle after destroying a loaded model no longer crashes in either binding.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, csharp
Domain
backend-api-design, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.