microsoft / microsoft/onnxruntime-genai

LLama-3.2-1B-Instruct Function Calling error

Open
#994 4 comments 0 reactions 0 assignees View on GitHub
platform:mobile
Dominant language
C++
Stars
1.1k
Forks
354
Avg merge
2d 16h
Merged PRs (30d)
85

Description

**Describe the bug**

When I use the FunctionCalling feature of the LLama-3.2-1B-Instruct model on an Android device, an error occurs.

The prompt is as follows:

```
<|begin_of_text|><|start_header_id|>system<|end_header_id|>

Environment: ipython
Cutting Knowledge Date: December 2023
Today Date: 18 Oct 2024

Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.<|eot_id|><|start_header_id|>user<|end_header_id|>

Given the following functions, please respond with a JSON for a function call with its proper arguments that best answers the given prompt.

Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.Do not use variables.

{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather for a location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The location to get the weather for, e.g. San Francisco, CA"
},
"format": {
"type": "string",
"description": "The format to return the weather in, e.g. 'celsius' or 'fahrenheit'",
"enum": [
"celsius",
"fahrenheit"
]
}
},
"required": [
"location",
"format"
]
}
}
}

What's the weather like today<|eot_id|><|start_header_id|>assistant<|end_header_id|>
```

The error message is as follows:

```
What is the weather today<|eot_id|><|start_header_id|>assistant<|end_header_id|>
02:09:04.367 libc A Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xb400006cc0dd2040 in tid 29979 (DefaultDispatch), pid 29931 (ative.app:genai)
02:09:04.737 DEBUG A Cmdline: ai.allama.generative.app:genai
02:09:04.737 DEBUG A pid: 29931, tid: 29979, name: DefaultDispatch >>> ai.allama.generative.app:genai <<<
02:09:04.737 DEBUG A #00 pc 0000000000b4531c /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG A #01 pc 0000000000e3952c /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG A #02 pc 0000000000e35b10 /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG A #03 pc 0000000000e32cf0 /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG A #04 pc 0000000000e328dc /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG A #05 pc 0000000000b43944 /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG A #06 pc 0000000000b40c48 /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG A #07 pc 0000000000cd46e8 /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG A #08 pc 0000000000cd37a4 /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG A #09 pc 0000000000cefb30 /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG A #10 pc 0000000000cd609c /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG A #11 pc 0000000000d0b788 /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG A #12 pc 0000000000d0b2b4 /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG A #13 pc 00000000004db874 /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG A #14 pc 00000000004ddb78 /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG A #15 pc 00000000004a7dfc /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG A #16 pc 0000000000147ccc /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime-genai.so (offset 0x178000) (Generators::State::Run(OrtSession&, OrtRunOptions&, int)+508) (BuildId: 3e67de3f11079c80638c65d55c883b0654441892)
02:09:04.737 DEBUG A #17 pc 0000000000138e28 /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime-genai.so (offset 0x178000) (Generators::DecoderOnly_State::Run(int, Generators::RoamingArray, Generators::RoamingArray)+140) (BuildId: 3e67de3f11079c80638c65d55c883b0654441892)
02:09:04.737 DEBUG A #18 pc 000000000012c190 /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime-genai.so (offset 0x178000) (Generators::Generator::ComputeLogits()+140) (BuildId: 3e67de3f11079c80638c65d55c883b0654441892)
02:09:04.737 DEBUG A #19 pc 000000000016ae7c /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime-genai.so (offset 0x178000) (OgaGenerator_ComputeLogits+28) (BuildId: 3e67de3f11079c80638c65d55c883b0654441892)
02:09:04.737 DEBUG A #20 pc 00000000000047bc /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime-genai-jni.so (offset 0x110000) (Java_ai_onnxruntime_genai_Generator_computeLogitsNative+20) (BuildId: a4099e8a78f869b68088fe2ce0e0af6aab84894a)
02:09:04.737 DEBUG A #31 pc 000000000000606c /data/data/ai.allama.generative.app/code_cache/.overlay/base.apk/classes6.dex (ai.allama.generative.service.onnx.OnnxModel.generateText+0)
02:09:04.737 DEBUG A #37 pc 0000000000004bf0 /data/data/ai.allama.generative.app/code_cache/.overlay/base.apk/classes6.dex (ai.allama.generative.service.onnx.OnnxChatCompletion$chat$1.invokeSuspend+0)
---------------------------- PROCESS ENDED (29931) for package ai.allama.generative.app ----------------------------
```

**Screenshots**

![Image](https://github.com/user-attachments/assets/9b921096-8837-40f2-8ad8-d176f00f594d)

**Desktop (please complete the following information):**
- OS: macOS M3
- Browser:chrome
- Version: 14.6.1

**Smartphone (please complete the following information):**
- Device: Android simulators
- OS: Android
- Browser: chrome
- Version :Android 15

**Additional context**

Everything works fine when I don't include FunctionCalling content in my normal use.

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the LLama-3.2-1B-Instruct function-calling prompt on an Android simulator and begin with OnnxModel.generateText and OnnxChatCompletion, using the libonnxruntime-genai stack frames around Generators::Generator::ComputeLogits. Trace the native crash through the reported call path; done means the prompt no longer causes SIGSEGV and function calling completes successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
android, cpp
Domain
ai-infra-agents, mobile-dev
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.