google / google/adk-python

A custom BaseLlm has no documented mapping from LlmRequest.config to a Vertex request body, and getting it wrong drops your tools with no error

Offen
#6,880 5 Kommentare 0 Reaktionen 1 zugewiesene Person Beansprucht von @surajksharma07 Auf GitHub ansehen
models
Vorherrschende Sprache
Python
Sterne
21.5k
Forks
4k
Ø Merge
1 T. 14 Std.
Gemergte PRs (30 T.)
37

Beschreibung

`LlmRequest.config` is a `types.GenerateContentConfig`. It holds 35 fields as one flat namespace. Those 35 fields go to three different places in a Vertex `generateContent` request, and nothing on the type says which field goes where.

Get the split wrong and nothing raises. The call returns 200. You get a normal looking response with normal looking text in it. Your tools are just gone, so the agent answers from memory instead of calling anything.

We lost most of a day to that, and we lost it in the wrong place. The agent looked like it was refusing to use its tools, and "the model won't call the tool" is almost always a prompt problem. So we went and rewrote the prompt. Then the tool descriptions. Neither was wrong.

## The three destinations

Sorted from the installed package, `google-adk` 2.7.1 with `google-genai` 2.x:

tools top level
tool_config top level
system_instruction top level
safety_settings top level
cached_content top level
labels top level

http_options client library only, rejected on the wire
automatic_function_calling client library only, rejected on the wire
should_return_http_response client library only, rejected on the wire

the other 26 inside generationConfig

Six, three and twenty six. A `BaseLlm` implementer has to know that split before writing a line, and there is nowhere to look it up.

## Repro

No network, no project, no credentials. It just shows the shape.

```python
from google.adk.models.llm_request import LlmRequest
from google.genai import types

cfg = types.GenerateContentConfig(
temperature=0.2,
max_output_tokens=8192,
system_instruction="You are a research agent.",
tools=[types.Tool(function_declarations=[
types.FunctionDeclaration(
name="a_tool",
description="Look up the weather in a city.",
parameters=types.Schema(
type=types.Type.OBJECT,
properties={"city": types.Schema(type=types.Type.STRING)},
),
)
])],
http_options=types.HttpOptions(timeout=60_000),
)
req = LlmRequest(model="gemini-3.5-flash", contents=[], config=cfg)

dumped = req.config.model_dump(mode="json", by_alias=True, exclude_none=True)
print(sorted(dumped))

# The obvious mapping. It is wrong, and it is wrong quietly.
body = {"contents": [], "generationConfig": dumped}
print("tools at top level?", "tools" in body)
print("tools buried in generationConfig?", "tools" in body["generationConfig"])
```

Output:

```
['httpOptions', 'maxOutputTokens', 'systemInstruction', 'temperature', 'tools']
tools at top level? False
tools buried in generationConfig? True
```

`tools` and `temperature` come out as siblings. They are not siblings anywhere on the wire. Send that body to Vertex and you get a 200 back with no function call in it, ever, because you never offered one.

## Why it is hard to find

Every signal points away from the real cause.

The response is well formed. There is no warning, no unknown field error, no empty candidate. The model is behaving correctly. It was asked a question with no tools attached, so it answered the question.

And a missing tool call looks exactly like a model that decided not to call the tool, which is a thing models genuinely do. So you go and argue with the prompt. That is the trap. The failure presents as a judgement call by the model when it is actually a serialization bug three layers down.

`http_options` at least fails loudly. Forward it and Vertex rejects the request as having an unknown field, so you find that one in a minute. It is the same underlying gap, just pointing the other way: some of this object is for the library and some is for the server, and nothing marks the boundary.

## What we do

A named list, because a guess is what got us here:

```python
TOP_LEVEL = {"tools", "toolConfig", "systemInstruction", "safetySettings",
"cachedContent", "labels"}

NOT_FOR_THE_WIRE = {"httpOptions", "automaticFunctionCalling",
"shouldReturnHttpResponse"}
```

That list is hand made from watching things fail, and it will rot. `GenerateContentConfig` gains fields. When it gains a top level one, every custom `BaseLlm` in the wild silently starts dropping it, and nobody finds out until an agent quietly stops using a feature.

## What would help

Either of these would have saved the day, and the first one is cheap.

A table in the `BaseLlm` docs saying which config fields are top level, which are client side, and which go in `generationConfig`. Just the list above, maintained next to the type that changes.

Or a helper that does the split, something like `llm_request.to_generate_content_body()`, so the mapping lives in one place and every implementer gets it right by default. That is the version that does not rot.

## This is not the same as #6518

Worth saying, because it looks adjacent. In #6518 the answer was that not every field in `GenerateContentConfig` is compatible with `LlmAgent`, and that is fair.

This is a different question. Nothing here is incompatible with anything. `tools` is supported, wanted, and central. ADK builds it into the config itself and then hands the object to code it expects to serialize it. The gap is that ADK never says how. Related in species to #6513 too, where a field went missing without a sound.

## Environment

`google-adk` 2.7.1, `google-genai` 2.x, Python 3.12, Vertex REST, custom `BaseLlm` over `InMemoryRunner`, a few hundred agent runs. We use a custom `BaseLlm` because every model call in our system goes through one retry and backoff path, and the agent is the chattiest caller we have.

Happy to put up the docs table as a PR, or the helper with unit tests if that is the direction you prefer. Say which and I will write it.

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.