ROCm / ROCm/FastFlowLM

Flm with gemma 4 has very poor ability to call tools

Open
#721 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
1.9k
Forks
152
Avg merge
4h 14m
Merged PRs (30d)
11

Description

I am using flm backend for lemonade server on Linux with Gemma 4 e4b.
And it's support for tools is very poor.

AMD Ryzen AI 9 HX PRO 375 w/ Radeon 890M × 12
Linux Mint 22.3 - Cinnamon 64-bit
7.0.0-31-generic
lemonade version 11.9.0
flm npu v1.0.3
gemma4-it-e4b-FLM 9.10GB


User

Please write a python script that writes json with "name" and "description" keys into out.txt file. Please save this python script into scrypt.py


Assistant (Coder · gemma4-it-e4b-FLM (local) · 11.9s)

Tool: write

Input:

{
  "content": "\"\"\"import json\n\ndata = {\n    \"name\": \"Example Item\",\n    \"description\": \"This is a sample description.\"\n}\n\nwith open('out.txt', 'w') as f:\n    json.dump(data, f)\n\"\"",
  "filePath": "\"scrypt.py\""
}

Output:

Wrote file successfully.

It wrote file with ["scrypt.py"] name and following invalid content:

"""import json

data = {
"name": "Example Item",
"description": "This is a sample description."
}

with open('out.txt', 'w') as f:
json.dump(data, f)
"

And later just stuck with tool calls:

Tool: write

Input:

{
  "content": "import json\n\ndata = {\n    \"name\": \"Example Item",
  "description": "This is a sample description."
}

Error:

The write tool was called with invalid arguments: SchemaError(Missing key
  at ["filePath"]).
Please rewrite the input so it satisfies the expected schema.

Assistant (Coder · gemma4-it-e4b-FLM (local) · 13.5s)

Tool: write

Input:

{
  "content": "import json\n\ndata = {\n    \"name\": \"Example Item",
  "description": "This is a sample description."
}

Error:

The write tool was called with invalid arguments: SchemaError(Missing key
  at ["filePath"]).
Please rewrite the input so it satisfies the expected schema.

response:write{value:<|"|>Wrote file successfully.<|"|>}<tool_call|>


Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the reported prompt with the flm backend, Gemma 4 e4b-FLM, lemonade 11.9.0, and flm npu v1.0.3 on Linux. Start by tracing the tool-call and write-tool request handling; done means the generated call preserves valid content and the required filePath, completes without schema errors, and creates the requested script.

Written by the indexing model from the issue text.

Assessment

Tech stack
linux
Domain
ai, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.