google / google/adk-python

[LiteLLM] role='tool_responses' for Gemma-4 models breaks tool calling on LM Studio (strict OpenAI role validation)

Offen
#6,482 9 Kommentare 0 Reaktionen 1 zugewiesene Person Beansprucht von @surajksharma07 Auf GitHub ansehen
models request clarification
Vorherrschende Sprache
Python
Sterne
21.5k
Forks
4k
Ø Merge
1 T. 14 Std.
Gemergte PRs (30 T.)
37

Beschreibung

**Describe the Bug**

`_content_to_message_param` in `google/adk/models/lite_llm.py` forces
`role="tool_responses"` for the tool-result message whenever the model
name matches `_is_gemma4_model` (regex `gemma-?4`), regardless of which
serving backend is actually handling the request:

```python
tool_role = "tool_responses" if _is_gemma4_model(model) else "tool"
```

This was introduced for Ollama (#5650) and broadened to also match
hyphenated `gemma-4` names to cover vLLM/llama.cpp (#6334). Both of
those fixes assume every llama.cpp-family server passes the `role`
value straight through to the model's chat template without validating
it against the standard OpenAI role enum.

**LM Studio does not** — its OpenAI-compatible endpoint validates
`messages[].role` strictly and rejects `tool_responses` outright, even
though LM Studio also serves GGUF models via a llama.cpp-based runtime.
The very backend #6334 was broadened to support (\"vLLM, llama.cpp\")
turns out not to be uniform: LM Studio's own compatibility layer sits
in front of llama.cpp and enforces the standard role set before the
model's chat template ever sees the message.

Net effect: any ADK agent with at least one tool, running a Gemma-4
model through LiteLLM against LM Studio, fails on the very first
tool-result turn.

**Steps to Reproduce**

1. Load any Gemma-4-named model in LM Studio (tested with
`google/gemma-4-e4b`) and start its local server
(`http://127.0.0.1:1234/v1`, OpenAI-compatible).
2. Run the following minimal ADK agent, which has a single trivial
tool unrelated to any specific application:

```python
import asyncio
from google.adk.agents import Agent
from google.adk.models.lite_llm import LiteLlm
from google.adk.runners import InMemoryRunner
from google.adk.tools.function_tool import FunctionTool
from google.genai import types

def roll_dice() -> dict:
"""Roll a six-sided die.

Returns:
A dict with the result.
"""
return {"result": 4}

model = LiteLlm(
model="openai/google/gemma-4-e4b",
api_base="http://127.0.0.1:1234/v1",
api_key="not-needed",
)
agent = Agent(name="t", model=model, instruction="Be helpful.", tools=[FunctionTool(roll_dice)])

async def main():
runner = InMemoryRunner(agent, app_name="t")
session = await runner.session_service.create_session(app_name="t", user_id="u")
message = types.Content(role="user", parts=[types.Part(text="Roll the dice for me")])
async for event in runner.run_async(user_id="u", session_id=session.id, new_message=message):
if event.is_final_response() and event.content and event.content.parts:
print(event.content.parts[0].text)

asyncio.run(main())
```

**Expected Behavior**

The agent calls `roll_dice`, receives the result, and produces a final
text response.

**Observed Behavior**

The first request (tool declaration only, no prior tool call) succeeds
normally and the model correctly emits a `tool_calls` response. The
*second* request — sent after the tool runs, carrying the tool result —
is rejected by LM Studio:

```
2026-07-25 21:51:45 [ERROR]
[google/gemma-4-e4b] Invalid 'content': 'content' field must be a string or an array of objects.
```

LM Studio's own debug log shows the exact outgoing message that trips
this:

```json
{
"role": "assistant",
"tool_calls": [
{
"type": "function",
"id": "200618556",
"function": { "name": "roll_dice", "arguments": "{}" }
}
]
},
{
"role": "tool_responses",
"tool_call_id": "200618556",
"content": "{\"result\": 4}"
}
```

`role: "tool_responses"` is not a role LM Studio's OpenAI-compatible
validator accepts (only the standard `system`/`user`/`assistant`/`tool`
set is), so the request as a whole is rejected. The reported error text
("`content` field must be...") is misleading — the actual problem is
the unrecognized `role` value, not the `content` field.

**Environment**

- `google-adk`: 2.5.0
- `litellm`: 1.93.0
- LM Studio: OpenAI-compatible server at `/v1`, `google/gemma-4-e4b`
(GGUF, llama.cpp-based runtime)
- Python: 3.13
- OS: Windows

**Additional Context**

Related: #5650 (Ollama, original fix), #6334 (broadened to hyphenated
`gemma-4` naming for vLLM/llama.cpp). This report is about a case
neither of those covers: a llama.cpp-based server (LM Studio) that
*does* validate the OpenAI role enum strictly, unlike whatever
vLLM/llama.cpp setup #6334 was tested against.

Possible directions for a fix (not prescriptive, just what occurred to
us):
- Make the tool-result role a `LiteLlm` constructor option instead of
an unconditional model-name-based inference, so callers can opt out
per backend.
- Detect the actual serving backend (e.g. via a capability probe, or a
`custom_llm_provider`/base-url heuristic) rather than matching on
model name alone, since "serves a `gemma-4`-named GGUF" does not
imply "accepts a non-standard `role` value."
- At minimum, catch the resulting `BadRequestError` and retry once with
the standard `role="tool"`, so a stricter backend degrades instead of
hard-failing.

Happy to provide more logs or test a patch against our setup if useful.

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.