microsoft / microsoft/onnxruntime-genai
[Bug?] library crash at runtime when loading the model
- Dominant language
- C++
- Stars
- 1.1k
- Forks
- 354
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 85
Description
Hi, I'm not sure if this is due to unsupported version of the runtime or because I've set it up wrong, I'm using unity and I'm not sure if I've done it wrong or because unity doesn't support .net 8 yet .
I've used this code to see if the library is loaded :
```
Assembly genAIAssembly = null;
foreach (var asm in AppDomain.CurrentDomain.GetAssemblies())
{
if (asm.GetName().Name.IndexOf("OnnxRuntimeGenAI", StringComparison.OrdinalIgnoreCase) >= 0)
{
genAIAssembly = asm;
Debug.Log(genAIAssembly);
Debug.Log(genAIAssembly.GlobalAssemblyCache);
Debug.Log(asm.GetLoadedModules());
Debug.Log("reloading dll");
foreach (Type generic in asm.ExportedTypes)
{
Debug.Log(" supported features of subtype " + generic + " in domain : " + AppDomain.CurrentDomain.FriendlyName);
Debug.Log(asm.Location);
Debug.Log(asm.IsFullyTrusted);
Assembly.LoadFrom(asm.CodeBase);
Debug.Log(asm.EscapedCodeBase);
}```
and here's my initlizer for the model / tokenizer :
```
public Model model = @"modelpath" //same as _modelDir
static Tokenizer tokenizer;
private GeneratorParams generatorParams;
model = new Model(_modelDir);
generatorParams = new GeneratorParams(model);
generatorParams.SetSearchOption("max_length", (ulong)200);
generatorParams.SetSearchOption("temperature", 0.7f);
generatorParams.SetSearchOption("top_p", 0.9f);
generatorParams.SetSearchOption("top_k", (ulong)40);
generatorParams.SetSearchOption("do_sample", true)
if (tokenizer == null)
tokenizer = new Tokenizer(model);
var inputTokensSpan = tokenizer.Encode(prompt);
// some processing code
string generatedText = tokenizer.Decode(outputTokens); // output .
void OnDestroy()
{
model?.Dispose();
tokenizer?.Dispose();
}
```
but it seems that the tokenizer is always null ! even if I make sure that the model path is correct , readable and stored in GameAssetFolder (Streaming assets)
it seems
```Error initializing model/tokenizer: System.DllNotFoundException: onnxruntime-genai assembly: type: member:(null) at (wrapper managed-to-native) Microsoft.ML.OnnxRuntimeGenAI.NativeMethods.OgaCreateModel(byte[],intptr&) at Microsoft.ML.OnnxRuntimeGenAI.Model..ctor (System.String modelPath) [0x00015] in :0 at LLM.Start () [0x0008c] in D:\Unity\projects\ONNXGENAIMM\ONNXGENAIMM\Assets\LLM.cs:45 UnityEngine.Debug:LogError (object) LLM:Start () (at Assets/LLM.cs:52)```
is relevant here too however I do see this message but i'm not sure what it indicates is it a crash due to linking process / the error of modelpath or simply because unity doesn't support .net 8 and all of my native library despite being loadable at the editor it crash for this reason at runtime .
```DllNotFoundException: onnxruntime-genai assembly: type: member:(null)
Microsoft.ML.OnnxRuntimeGenAI.Model..ctor (System.String modelPath) (at <16bd269ae35248cf8809999f43fc3b78>:0)
MyNamespace.Test.LoadModel () (at Assets/scripts/test.cs:132)
UnityEngine.Debug:LogException(Exception)
MyNamespace.Test:LoadModel() (at Assets/scripts/test.cs:164)```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with Assets/LLM.cs at lines 45 and 52, then compare Assets/scripts/test.cs at lines 132 and 164. Reproduce the Unity runtime failure around the Model constructor and inspect the reported DllNotFoundException. Done means the model and tokenizer initialize successfully in the target runtime without the native library error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, unity
- Domain
- game-dev
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100