googleapis / googleapis/python-genai
Automatic function calling reports usage_metadata for the final model call only, so multi-hop requests undercount tokens
- Dominant language
- Python
- Stars
- 4k
- Forks
- 1k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 40
Description
### Summary
With automatic function calling enabled, a call that takes several tool
round trips returns `usage_metadata` for the **final** model call only.
The tokens spent on every earlier step are not reported anywhere, so any
cost or quota figure derived from `response.usage_metadata` undercounts
by however many hops the model took — silently, and by a factor that
varies per request.
### Where
`google/genai/models.py`, at `ccbc6c5`. The AFC loop reassigns `response`
each iteration and returns the last one:
```python
response = self._generate_content( # ~6604, per iteration
model=model, contents=contents, config=config_model
)
...
remaining_remote_calls_afc -= 1 # ~6624
if remaining_remote_calls_afc == 0:
logger.info('Reached max remote calls for automatic function calling.')
...
return response # ~6648 - the LAST call only
```
Nothing accumulates `usage_metadata` across iterations. The same shape
appears in the streaming path (~6830) and the async path (~8821).
### Why it matters
The undercount is invisible from the caller's side: you get a
well-formed `usage_metadata` with plausible numbers. A single-hop request
reports correctly, so the error only appears once the model starts using
tools — which is exactly when an agent's cost profile starts to matter.
Anything metering spend, enforcing budgets, or attributing cost per
request is wrong in the same direction every time.
### Reproduce
```python
from google import genai
from google.genai import types
CALLS = {"n": 0}
def get_weather(city: str) -> str:
CALLS["n"] += 1
return f"sunny in {city}"
client = genai.Client()
resp = client.models.generate_content(
model="gemini-2.5-flash",
contents="Compare the weather in Paris, Tokyo and Lima.",
config=types.GenerateContentConfig(tools=[get_weather]),
)
print("tool calls:", CALLS["n"])
print("prompt tokens:", resp.usage_metadata.prompt_token_count)
print("total tokens:", resp.usage_metadata.total_token_count)
```
Compare the reported totals against the number of round trips the model
actually made. The reported figure corresponds to one call, not the
sequence.
### Suggested fix
I have deliberately not sent a patch, because the useful part of this is
a decision I should not make for you:
1. **Aggregate in place** — sum the per-call metadata into the returned
`usage_metadata`. Simplest for callers, but it silently changes the
meaning of an existing field for anyone who currently reads it as
"the last call".
2. **Add a separate field** — e.g. a cumulative total alongside the
existing per-call value, leaving current semantics untouched.
3. **Expose the steps** — return per-call metadata for the whole AFC
chain, which also answers "which hop was expensive".
Happy to send a PR for whichever you prefer.
### Environment
- google-genai: read from source at `ccbc6c5` (current `main`); latest
release at time of writing is v2.17.0
- Python: 3.13
- Not a Vertex-vs-Gemini-API distinction — the loop is shared
Contributor guide
Assessment
This issue has not been assessed yet.