[Issue]: Gemma4 tool-call parsing produces wrong name and arguments (v1.0.5)

Open

#722 opened on Sep 10, 2026 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty: 3/5
Estimated time: 1-2 days
Newbie friendliness: 66/100

Issue type: Bug
Clarity: Mostly clear
Activity status: Active
Tech stack: cpp
Domain: ai-infra-agents, api, backend-api-design

Research direction

Start at the Gemma4 tool-call parsing used by the flm serve /v1/chat/completions entry point. Reproduce the supplied curl request, then add a regression test for the wrapper and verify that the response uses memory_search as the function name with only the inner arguments object.

Written by the indexing model from the issue text.

Description

Problem Description

Title: Gemma4 tool-call parsing produces wrong name and arguments (v1.0.5)

Body:

Hi FastFlowLM team,

First, thanks for the v1.0.5 release and for the recent improvements to Gemma4 tool-call argument parsing. Unfortunately, I'm still hitting a bug where tool calls are parsed incorrectly, and the resulting tool invocation is unusable by OpenAI-compatible clients (OpenClaw in my case).

Environment
FastFlowLM version: v1.0.5

Model: gemma4-it:12b (also reproducible with gemma4-it:e4b)

Server mode: flm serve gemma4-it:12b (OpenAI-compatible endpoint, /v1/chat/completions)

Client: OpenClaw 2026.9.3 (uses api=openai-completions, tool_choice: auto, stream: true)

Host: AMD Ryzen AI NPU (Strix Point)

What the model emits
The model produces a correct native Gemma4 tool call:

text
<|tool_call>call:tool_call{args:{query:"тренировки на пресс или ноги"},id:<|"|>memory_search<|"|>}<tool_call|>
Note: the actual function name is memory_search, and its argument is {"query": "..."}. The surrounding tool_call{...} wrapper is what Gemma4 emits, and the real tool identity lives in the id field.

What FLM parses it into
FLM's own log line shows the parsed result:

text
[FLM] Tool name: tool_call
[FLM] Tool args: {"args":{"query":"тренировки на пресс или ноги"},"id":"memory_search"}
[FLM] Tool JSON: {"name":"tool_call","arguments":"{"args":{"query":"тренировки на пресс или ноги"},"id":"memory_search"}"}
What the client receives
The OpenAI-compatible response contains:

json
{
"function": {
"name": "tool_call",
"arguments": "{"args":{"query":"тренировки на пресс или ноги"},"id":"memory_search"}"
},
"id": "call17890662170",
"type": "function"
}
Expected behavior
For the same model output, FLM should produce:

json
{
"function": {
"name": "memory_search",
"arguments": "{"query":"тренировки на пресс или ноги"}"
},
"id": "call17890662170",
"type": "function"
}
i.e.:

name should be the real tool name (memory_search), not the wrapper tool_call.

arguments should be the inner argument object ({"query": "..."}), not the whole {args, id} envelope.

Impact
Because the emitted name is tool_call (which does not exist in the client's tool list), the tool is never executed. The client sees an unknown function, drops the call, and the model then hallucinates / "negotiates" a response instead of returning real data. This completely breaks agentic tool use (memory search, web search, file operations, etc.) for Gemma4 models over the OpenAI-compatible endpoint.

Reproduction
Minimal reproduction with a single tool:

bash
curl http://127.0.0.1:52625/v1/chat/completions
-H "Content-Type: application/json"
-d '{
"model": "gemma4-it:12b",
"messages": [
{"role": "user", "content": "Find my press or leg workouts in memory"}
],
"tools": [
{
"type": "function",
"function": {
"name": "memory_search",
"description": "Search long-term memory",
"parameters": {
"type": "object",
"properties": {
"query": {"type": "string"}
},
"required": ["query"]
}
}
}
],
"tool_choice": "auto"
}'
The response will contain a tool call with name: "tool_call" instead of name: "memory_search".

Notes
I understand v1.0.5 improved Gemma4 tool-call argument parsing for edge cases. This bug looks like a separate issue: the function name mapping and the argument unwrapping are still wrong.

The same model, when served via a different OpenAI-compatible backend (e.g. llama.cpp with Qwen-family models), produces correct tool calls, so the model family and the client are not the problem — the parsing inside FLM is.

This was previously reported in the OpenClaw tracker as a Gemma4 tool-call recovery issue (openclaw#104868), but the fix needs to happen on the FLM side, since FLM is the component that produces the malformed OpenAI-format tool call.

Request
Could you please:

Map the Gemma4 call:{...} envelope so that name becomes the real tool name (from the id field) and arguments becomes the inner argument object.

Add a test case that covers this exact pattern, since it's the standard output shape for Gemma4 tool calls.

Happy to provide more logs or test a patched build if that helps.

Thanks!

Operating System

Linux Debian 13

CPU

24x AMD Ryzen AI 9 HX 370 w/ Radeon 890M

GPU

AMd Radeon 890M/XDNA RyzenAI-npu4

ROCm Version

ROCm 7.0.2 (rocm-core package)

Installation Method

Built from source

Installed ROCm Packages / Versions

amdgpu-install script (v7.0.2) with --usecase=rocm

wget https://repo.radeon.com/amdgpu-install/7.0.2/ubuntu/noble/amdgpu-install_7.0.2.70002-1_all.deb
dpkg -i amdgpu-install_7.0.2.70002-1_all.deb
amdgpu-install -y --usecase=rocm

ROCm Component

No response

Steps to Reproduce

2. Install ROCm

RUN echo "=== INSTALLING amdgpu-install ===" &&
apt-get update &&
apt-get install -y wget &&
wget -q https://repo.radeon.com/amdgpu-install/7.0.2/ubuntu/noble/amdgpu-install_7.0.2.70002-1_all.deb &&
dpkg -i amdgpu-install_7.0.2.70002-1_all.deb || true &&
apt-get install -f -y &&
echo "=== INSTALLING ROCm 7.0.2 via amdgpu-install ===" &&
amdgpu-install -y --usecase=rocm &&
apt-get clean && rm -rf /var/lib/apt/lists/* && rm amdgpu-install_*.deb

3. XRT

RUN mkdir -p /opt/xilinx/xrt/lib
/opt/xilinx/xrt/include/xrt/deprecated
/opt/xilinx/xrt/include/xrt/detail
/opt/xilinx/xrt/include/xrt/experimental

COPY xrt/xrt/lib/libxrt_coreutil.so* /opt/xilinx/xrt/lib/

4. FLM

COPY flm-real /opt/fastflowlm/bin/flm-real
COPY flm /opt/fastflowlm/bin/flm
COPY lib /opt/fastflowlm/lib
COPY model_list.json /opt/fastflowlm/bin/
COPY model_info.json /opt/fastflowlm/bin/

RUN chmod +x /opt/fastflowlm/bin/flm-real /opt/fastflowlm/bin/flm &&
ln -s /opt/fastflowlm/bin/flm /usr/local/bin/flm

5. vars

ENV PATH=/usr/local/bin:/usr/bin:/bin:/opt/fastflowlm/bin:$PATH
LD_LIBRARY_PATH=/opt/fastflowlm/lib:/opt/xilinx/xrt/lib:/usr/lib/x86_64-linux-gnu

WORKDIR /root/.config/flm

ENTRYPOINT ["flm", "serve", "--host", "0.0.0.0", "--port", "52625"]

(Optional for Linux users) Output of rocminfo --support

rocminfo --support output

Paste output here

Additional Information