googleapis / googleapis/python-genai
OpenAI compatibility layer: streaming returns finish_reason "stop" for tool calls; non-streaming returns "tool_calls" for the identical request
- Dominant language
- Python
- Stars
- 4k
- Forks
- 1k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 40
Description
**Note on venue:** this is an API-behavior report about the hosted OpenAI-compatibility layer (`generativelanguage.googleapis.com/v1beta/openai/`), reproduced with plain `curl` — no SDK involved. Filing here because this tracker actively triages Gemini API behavior issues; happy to move it wherever the team prefers. A forum thread reporting the same symptom ([discuss.ai.google.dev/t/112704](https://discuss.ai.google.dev/t/the-finish-reason-is-stop-instead-of-tool-calls-in-openai-compatible-endpoint/112704)) has a Google response requesting a full payload and repro — this issue provides exactly that.
## Summary
For the **identical request**, the OpenAI-compatibility layer disagrees with itself about `finish_reason` when the model responds with a tool call:
| Mode | Response contains | `finish_reason` |
|---|---|---|
| Non-streaming | `message.tool_calls` | `"tool_calls"` ✅ |
| Streaming (`stream: true`) | `tool_calls` delta chunk | `"stop"` ❌ |
Per OpenAI Chat Completions semantics, `finish_reason: "tool_calls"` is the signal that tells a client to execute tools and continue the loop. Agent frameworks built on the OpenAI SDK branch on this value, so in streaming mode the loop terminates instead of dispatching the tool — tool calling silently breaks, but only when streaming.
Reproduced **2026-08-17** on `gemini-3.7-flash` and `gemini-3.1-flash-lite-preview` (identical behavior on both, so this looks layer-wide rather than model-specific).
## Minimal repro
`payload.json`:
```json
{
"model": "gemini-3.7-flash",
"messages": [{"role": "user", "content": "What is the weather in Paris? Use the tool."}],
"tools": [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather for a city",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"]
}
}
}]
}
```
**Non-streaming:**
```bash
curl -s https://generativelanguage.googleapis.com/v1beta/openai/chat/completions \
-H "Authorization: Bearer $GEMINI_API_KEY" \
-H "Content-Type: application/json" \
-d @payload.json
```
Response (HTTP 200, `id: n3iCar2iBJTBqtsP2syK2Qo`, signatures truncated for readability):
```json
{
"choices": [{
"finish_reason": "tool_calls",
"index": 0,
"message": {
"role": "assistant",
"tool_calls": [{
"extra_content": {"google": {"thought_signature": "Ep0CCpoC…(truncated)"}},
"function": {"arguments": "{\"city\":\"Paris\"}", "name": "get_weather"},
"id": "call_482567",
"type": "function"
}]
}
}],
"created": 1786935455,
"model": "gemini-3.7-flash",
"object": "chat.completion",
"usage": {"completion_tokens": 16, "prompt_tokens": 55, "total_tokens": 117}
}
```
**Streaming** — same payload plus `"stream": true, "stream_options": {"include_usage": true}`:
```bash
curl -s https://generativelanguage.googleapis.com/v1beta/openai/chat/completions \
-H "Authorization: Bearer $GEMINI_API_KEY" \
-H "Content-Type: application/json" \
-d @payload_stream.json
```
Full SSE output (HTTP 200, `id: oXiCaoOBBsPg8QGa35aRCw`, signatures truncated):
```
data: {"choices":[{"delta":{"role":"assistant","tool_calls":[{"extra_content":{"google":{"thought_signature":"EpwCCpkC…(truncated)"}},"function":{"arguments":"{\"city\":\"Paris\"}","name":"get_weather"},"id":"call_1839541","type":"function"}]},"index":0}],"created":1786935457,"id":"oXiCaoOBBsPg8QGa35aRCw","model":"gemini-3.7-flash","object":"chat.completion.chunk","usage":{...}}
data: {"choices":[{"delta":{"role":"assistant"},"finish_reason":"stop","index":0}],"created":1786935457,"id":"oXiCaoOBBsPg8QGa35aRCw","model":"gemini-3.7-flash","object":"chat.completion.chunk","usage":{...}}
data: [DONE]
```
The tool call arrives complete (id + name + full arguments) in a single delta chunk; the terminal chunk then reports `finish_reason: "stop"` even though the only content produced was a tool call. Expected: `"tool_calls"`, matching the non-streaming response for the same request.
## Impact / ecosystem context
Because the non-streaming path already returns `"tool_calls"`, this looks like a bug in the streaming path of the compat layer rather than an intentional mapping. Downstream, several projects now carry independent normalization patches for it, e.g. [BerriAI/litellm#21041](https://github.com/BerriAI/litellm/issues/21041), [BerriAI/litellm#12249](https://github.com/BerriAI/litellm/issues/12249), and agent-side breakage reports like [anomalyco/opencode#14972](https://github.com/anomalyco/opencode/issues/14972); OpenRouter also rewrites the value at their edge.
We run an OpenAI-compatible gateway (clawapi.org) that routes to Gemini among other upstreams; we have already shipped the same normalization on our side (stream-side `stop` → `tool_calls` when tool-call deltas were emitted), so this report is not blocking us — filing it so the fix can land at the source and the ecosystem can eventually drop these patches.
## Related observation (separate, minor)
In the same streaming capture, entries inside the `tool_calls` delta array carry no `index` field (OpenAI clients use it to assemble parallel tool calls; some SDK codepaths `KeyError` on it). Previously reported on the forum in Jan 2025: [discuss.ai.google.dev/t/59886](https://discuss.ai.google.dev/t/gemini-openai-compatibility-issue-with-tool-call-streaming/59886). Mentioning for completeness since the capture above shows it; can file separately if useful.
Contributor guide
Assessment
This issue has not been assessed yet.