microsoft / microsoft/onnxruntime-genai
Crash when loading a model the 2nd time after recreating OgaHandle
- Dominant language
- C++
- Stars
- 1.1k
- Forks
- 354
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 85
Description
## Describe the bug
Creating an `OgaHandle`, loading a model, deleting both of them, and creating a new `OgaHandle` causes a crash. This happens in both C# and C++.
## To Reproduce
Steps to reproduce the behavior:
1. Create an `OgaHandle`
2. Load any model (I'm using [this one](https://github.com/microsoft/onnxruntime-genai/tree/main/test/test_models/hf-internal-testing/tiny-random-gpt2-fp32) with a chat template manually added to `tokenizer_config.json`)
3. Destroy or dispose the model
4. Destroy or dispose the `OgaHandle`
5. Create a new `OgaHandle`
6. Load any model again
The crash doesn't happen if a model isn't loaded before destroying the `OgaHandle`. It also happens with the Deepseek Qwen distills and it also doesn't matter if the model was used for generation.
You may use the code in [this repository](https://github.com/Zagusan/OgaHandle-Repro).
## Expected behavior
Since `OgaHandle` is essential for cleaning up resources, it should allow for loading another model after destroying and recreating it
## Environment
- OS: Windows 11 23H2
- Device: CPU
- Tested OGA versions: 0.11.2, 0.11.3, 0.11.4, and 0.12.2 (downloaded through NuGet)
- Tested ONNX Runtime version: 1.23.2 and 1.24.1
## Additional context
### C# error:
```
Exception thrown at 0x00007FFFC52F593C (onnxruntime-genai.dll) in OgaHandleRepro.exe: 0xC0000005: Access violation reading location 0x0000000000000000.
```
### C# stack trace:
```
onnxruntime-genai.dll!00007fff9dcb593c() Unknown
onnxruntime-genai.dll!00007fff9dd10d2b() Unknown
[Managed to Native Transition]
Microsoft.ML.OnnxRuntimeGenAI.dll!Microsoft.ML.OnnxRuntimeGenAI.Model.Model(Microsoft.ML.OnnxRuntimeGenAI.Config config) Unknown
OgaHandleRepro.dll!Repro.Program.LoadRandomGPT2() Line 33 C#
OgaHandleRepro.dll!Repro.Program.Main(string[] args) Line 55 C#
[Native to Managed Transition]
hostpolicy.dll!00007fffa015da9a() Unknown
hostpolicy.dll!00007fffa015dd3c() Unknown
hostpolicy.dll!00007fffa015ecb1() Unknown
hostfxr.dll!00007fffa449d56b() Unknown
hostfxr.dll!00007fffa44a029c() Unknown
hostfxr.dll!00007fffa44a2676() Unknown
hostfxr.dll!00007fffa44a079d() Unknown
hostfxr.dll!00007fffa4498998() Unknown
OgaHandleRepro.exe!00007ff7c7b12dec() Unknown
OgaHandleRepro.exe!00007ff7c7b13256() Unknown
OgaHandleRepro.exe!00007ff7c7b22548() Unknown
kernel32.dll!00007ff89380259d() Unknown
ntdll.dll!00007ff894a8af78() Unknown
```
### C++ error:
```
Unhandled exception at 0x00007FF89204055C in OgaHandleRepro.exe: Microsoft C++ exception: std::runtime_error at memory location 0x00000068F391F4D8.
```
The C++ error changed slightly in version 0.11.3 and beyond. Now, both C# and C++have the same access violation error:
```
Exception thrown at 0x00007FFB86A9720C (onnxruntime-genai.dll) in OgaHandleRepro.exe: 0xC0000005: Access violation reading location 0x0000000000000000.
```
### C++ stack trace:
```
onnxruntime-genai.dll!00007fffaad4593c() Unknown
onnxruntime-genai.dll!00007fffaada0d2b() Unknown
OgaHandleRepro.exe!OgaModel::Create(const OgaConfig & config) Line 240 C++
OgaHandleRepro.exe!LoadRandomGPT2() Line 33 C++
OgaHandleRepro.exe!main() Line 54 C++
OgaHandleRepro.exe!invoke_main() Line 79 C++
OgaHandleRepro.exe!__scrt_common_main_seh() Line 288 C++
OgaHandleRepro.exe!__scrt_common_main() Line 331 C++
OgaHandleRepro.exe!mainCRTStartup(void * __formal) Line 17 C++
kernel32.dll!00007ff89380259d() Unknown
ntdll.dll!00007ff894a8af78() Unknown
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the lifecycle in the linked OgaHandle-Repro repository, then inspect OgaModel::Create at line 240 and the C# Model.Model constructor shown in the stack traces. Compare the first and second OgaHandle/model loads in both C++ and C#. Done means recreating the handle after destroying a loaded model no longer crashes in either binding.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, csharp
- Domain
- backend-api-design, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100