openvinotoolkit / openvinotoolkit/model_server

OVMS tools call for Gemma 4 E4B failed

Open
#4,586 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
931
Forks
277
Avg merge
2d 13h
Merged PRs (30d)
68

Description

I am setting up an OVMS for Linux, OpenCode and Gemma 4 E4B.
And I see that tools call request has some another tool call format:

Expected:

{ "tool_calls": [ { "type": "function", "function": { "name": "write", "arguments": "{"content": "TEXT", "filePath": "/absolute/path/file.txt"}" } } ] }

Got:

User

Try to save some small file first

Assistant (Build · ovms.gemma-4-e4b (local) · 7.1s)

{
"tool_calls": [
{
"name": "write",
"arguments": {
"filePath": "small_file.txt",
"content": "This is a small test file."
}
}
]
}

As you can see there are some differences in JSON. Is there a way to change/customize this without code change?

How do I run ovms:

ovms --model_name gemma-4-e4b
--model_path ./gemma-4-e4b-ov-int4
--target_device $DEVICE
--port 9080
--rest_port 8080
--cache_dir ./gemma-4-e4b-ov-int4/npu_cache
--log_level $LOG_LEVEL
--tool_parser gemma4

My config:

Ubuntu 24.04.5 LTS
6.17.0-1032-oem
Intel© Core™ Ultra 7 265H × 16

OpenVINO Model Server 2026.3.0.6f3df706b

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the OVMS tool-parser behavior selected by --tool_parser gemma4 and compare its response with the expected tool_calls format shown in the report. Reproduce the request using Gemma 4 E4B on Linux, determine whether configuration can change the format, and consider the issue done when the returned schema matches the expected format or the customization limitation is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, linux
Domain
ai, api
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.