google / google/adk-python

[LiteLLM] role='tool_responses' for Gemma-4 models breaks tool calling on LM Studio (strict OpenAI role validation)

Abierto
#6,482 9 comentarios 0 reacciones 1 asignado Reclamado por @surajksharma07 Ver en GitHub
models request clarification
Lenguaje dominante
Python
Estrellas
21.5k
Forks
4k
Merge medio
1 d 14 h
PR fusionados (30 d)
37

Descripción

**Describe the Bug**

`_content_to_message_param` in `google/adk/models/lite_llm.py` forces
`role="tool_responses"` for the tool-result message whenever the model
name matches `_is_gemma4_model` (regex `gemma-?4`), regardless of which
serving backend is actually handling the request:

```python
tool_role = "tool_responses" if _is_gemma4_model(model) else "tool"
```

This was introduced for Ollama (#5650) and broadened to also match
hyphenated `gemma-4` names to cover vLLM/llama.cpp (#6334). Both of
those fixes assume every llama.cpp-family server passes the `role`
value straight through to the model's chat template without validating
it against the standard OpenAI role enum.

**LM Studio does not** — its OpenAI-compatible endpoint validates
`messages[].role` strictly and rejects `tool_responses` outright, even
though LM Studio also serves GGUF models via a llama.cpp-based runtime.
The very backend #6334 was broadened to support (\"vLLM, llama.cpp\")
turns out not to be uniform: LM Studio's own compatibility layer sits
in front of llama.cpp and enforces the standard role set before the
model's chat template ever sees the message.

Net effect: any ADK agent with at least one tool, running a Gemma-4
model through LiteLLM against LM Studio, fails on the very first
tool-result turn.

**Steps to Reproduce**

1. Load any Gemma-4-named model in LM Studio (tested with
`google/gemma-4-e4b`) and start its local server
(`http://127.0.0.1:1234/v1`, OpenAI-compatible).
2. Run the following minimal ADK agent, which has a single trivial
tool unrelated to any specific application:

```python
import asyncio
from google.adk.agents import Agent
from google.adk.models.lite_llm import LiteLlm
from google.adk.runners import InMemoryRunner
from google.adk.tools.function_tool import FunctionTool
from google.genai import types

def roll_dice() -> dict:
"""Roll a six-sided die.

Returns:
A dict with the result.
"""
return {"result": 4}

model = LiteLlm(
model="openai/google/gemma-4-e4b",
api_base="http://127.0.0.1:1234/v1",
api_key="not-needed",
)
agent = Agent(name="t", model=model, instruction="Be helpful.", tools=[FunctionTool(roll_dice)])

async def main():
runner = InMemoryRunner(agent, app_name="t")
session = await runner.session_service.create_session(app_name="t", user_id="u")
message = types.Content(role="user", parts=[types.Part(text="Roll the dice for me")])
async for event in runner.run_async(user_id="u", session_id=session.id, new_message=message):
if event.is_final_response() and event.content and event.content.parts:
print(event.content.parts[0].text)

asyncio.run(main())
```

**Expected Behavior**

The agent calls `roll_dice`, receives the result, and produces a final
text response.

**Observed Behavior**

The first request (tool declaration only, no prior tool call) succeeds
normally and the model correctly emits a `tool_calls` response. The
*second* request — sent after the tool runs, carrying the tool result —
is rejected by LM Studio:

```
2026-07-25 21:51:45 [ERROR]
[google/gemma-4-e4b] Invalid 'content': 'content' field must be a string or an array of objects.
```

LM Studio's own debug log shows the exact outgoing message that trips
this:

```json
{
"role": "assistant",
"tool_calls": [
{
"type": "function",
"id": "200618556",
"function": { "name": "roll_dice", "arguments": "{}" }
}
]
},
{
"role": "tool_responses",
"tool_call_id": "200618556",
"content": "{\"result\": 4}"
}
```

`role: "tool_responses"` is not a role LM Studio's OpenAI-compatible
validator accepts (only the standard `system`/`user`/`assistant`/`tool`
set is), so the request as a whole is rejected. The reported error text
("`content` field must be...") is misleading — the actual problem is
the unrecognized `role` value, not the `content` field.

**Environment**

- `google-adk`: 2.5.0
- `litellm`: 1.93.0
- LM Studio: OpenAI-compatible server at `/v1`, `google/gemma-4-e4b`
(GGUF, llama.cpp-based runtime)
- Python: 3.13
- OS: Windows

**Additional Context**

Related: #5650 (Ollama, original fix), #6334 (broadened to hyphenated
`gemma-4` naming for vLLM/llama.cpp). This report is about a case
neither of those covers: a llama.cpp-based server (LM Studio) that
*does* validate the OpenAI role enum strictly, unlike whatever
vLLM/llama.cpp setup #6334 was tested against.

Possible directions for a fix (not prescriptive, just what occurred to
us):
- Make the tool-result role a `LiteLlm` constructor option instead of
an unconditional model-name-based inference, so callers can opt out
per backend.
- Detect the actual serving backend (e.g. via a capability probe, or a
`custom_llm_provider`/base-url heuristic) rather than matching on
model name alone, since "serves a `gemma-4`-named GGUF" does not
imply "accepts a non-standard `role` value."
- At minimum, catch the resulting `BadRequestError` and retry once with
the standard `role="tool"`, so a stricter backend degrades instead of
hard-failing.

Happy to provide more logs or test a patch against our setup if useful.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.