microsoft / microsoft/onnxruntime-genai

[Bug?] library crash at runtime when loading the model

Open
#2,023 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
1.1k
Forks
354
Avg merge
2d 16h
Merged PRs (30d)
85

Description

Hi, I'm not sure if this is due to unsupported version of the runtime or because I've set it up wrong, I'm using unity and I'm not sure if I've done it wrong or because unity doesn't support .net 8 yet .

I've used this code to see if the library is loaded :

Image

```

Assembly genAIAssembly = null;
foreach (var asm in AppDomain.CurrentDomain.GetAssemblies())
{
if (asm.GetName().Name.IndexOf("OnnxRuntimeGenAI", StringComparison.OrdinalIgnoreCase) >= 0)
{
genAIAssembly = asm;

Debug.Log(genAIAssembly);
Debug.Log(genAIAssembly.GlobalAssemblyCache);
Debug.Log(asm.GetLoadedModules());
Debug.Log("reloading dll");

foreach (Type generic in asm.ExportedTypes)
{
Debug.Log(" supported features of subtype " + generic + " in domain : " + AppDomain.CurrentDomain.FriendlyName);
Debug.Log(asm.Location);
Debug.Log(asm.IsFullyTrusted);
Assembly.LoadFrom(asm.CodeBase);
Debug.Log(asm.EscapedCodeBase);
}```

and here's my initlizer for the model / tokenizer :

```
public Model model = @"modelpath" //same as _modelDir
static Tokenizer tokenizer;
private GeneratorParams generatorParams;

model = new Model(_modelDir);

generatorParams = new GeneratorParams(model);
generatorParams.SetSearchOption("max_length", (ulong)200);
generatorParams.SetSearchOption("temperature", 0.7f);
generatorParams.SetSearchOption("top_p", 0.9f);
generatorParams.SetSearchOption("top_k", (ulong)40);
generatorParams.SetSearchOption("do_sample", true)

if (tokenizer == null)
tokenizer = new Tokenizer(model);
var inputTokensSpan = tokenizer.Encode(prompt);
// some processing code

string generatedText = tokenizer.Decode(outputTokens); // output .

void OnDestroy()
{
model?.Dispose();
tokenizer?.Dispose();
}

```
but it seems that the tokenizer is always null ! even if I make sure that the model path is correct , readable and stored in GameAssetFolder (Streaming assets)

it seems

```Error initializing model/tokenizer: System.DllNotFoundException: onnxruntime-genai assembly: type: member:(null) at (wrapper managed-to-native) Microsoft.ML.OnnxRuntimeGenAI.NativeMethods.OgaCreateModel(byte[],intptr&) at Microsoft.ML.OnnxRuntimeGenAI.Model..ctor (System.String modelPath) [0x00015] in :0 at LLM.Start () [0x0008c] in D:\Unity\projects\ONNXGENAIMM\ONNXGENAIMM\Assets\LLM.cs:45 UnityEngine.Debug:LogError (object) LLM:Start () (at Assets/LLM.cs:52)```

is relevant here too however I do see this message but i'm not sure what it indicates is it a crash due to linking process / the error of modelpath or simply because unity doesn't support .net 8 and all of my native library despite being loadable at the editor it crash for this reason at runtime .

```DllNotFoundException: onnxruntime-genai assembly: type: member:(null)
Microsoft.ML.OnnxRuntimeGenAI.Model..ctor (System.String modelPath) (at <16bd269ae35248cf8809999f43fc3b78>:0)
MyNamespace.Test.LoadModel () (at Assets/scripts/test.cs:132)
UnityEngine.Debug:LogException(Exception)
MyNamespace.Test:LoadModel() (at Assets/scripts/test.cs:164)```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with Assets/LLM.cs at lines 45 and 52, then compare Assets/scripts/test.cs at lines 132 and 164. Reproduce the Unity runtime failure around the Model constructor and inspect the reported DllNotFoundException. Done means the model and tokenizer initialize successfully in the target runtime without the native library error.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, unity
Domain
game-dev
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.