googleapis / googleapis/python-genai

OpenAI compatibility layer: streaming returns finish_reason "stop" for tool calls; non-streaming returns "tool_calls" for the identical request

Open
#2,868 1 comment 2 reactions 1 assignee Assigned to @kkorpal View on GitHub
priority: p2 type: bug
Dominant language
Python
Stars
4k
Forks
1k
Avg merge
2d 11h
Merged PRs (30d)
40

Description

**Note on venue:** this is an API-behavior report about the hosted OpenAI-compatibility layer (`generativelanguage.googleapis.com/v1beta/openai/`), reproduced with plain `curl` — no SDK involved. Filing here because this tracker actively triages Gemini API behavior issues; happy to move it wherever the team prefers. A forum thread reporting the same symptom ([discuss.ai.google.dev/t/112704](https://discuss.ai.google.dev/t/the-finish-reason-is-stop-instead-of-tool-calls-in-openai-compatible-endpoint/112704)) has a Google response requesting a full payload and repro — this issue provides exactly that.

## Summary

For the **identical request**, the OpenAI-compatibility layer disagrees with itself about `finish_reason` when the model responds with a tool call:

| Mode | Response contains | `finish_reason` |
|---|---|---|
| Non-streaming | `message.tool_calls` | `"tool_calls"` ✅ |
| Streaming (`stream: true`) | `tool_calls` delta chunk | `"stop"` ❌ |

Per OpenAI Chat Completions semantics, `finish_reason: "tool_calls"` is the signal that tells a client to execute tools and continue the loop. Agent frameworks built on the OpenAI SDK branch on this value, so in streaming mode the loop terminates instead of dispatching the tool — tool calling silently breaks, but only when streaming.

Reproduced **2026-08-17** on `gemini-3.7-flash` and `gemini-3.1-flash-lite-preview` (identical behavior on both, so this looks layer-wide rather than model-specific).

## Minimal repro

`payload.json`:

```json
{
"model": "gemini-3.7-flash",
"messages": [{"role": "user", "content": "What is the weather in Paris? Use the tool."}],
"tools": [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather for a city",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"]
}
}
}]
}
```

**Non-streaming:**

```bash
curl -s https://generativelanguage.googleapis.com/v1beta/openai/chat/completions \
-H "Authorization: Bearer $GEMINI_API_KEY" \
-H "Content-Type: application/json" \
-d @payload.json
```

Response (HTTP 200, `id: n3iCar2iBJTBqtsP2syK2Qo`, signatures truncated for readability):

```json
{
"choices": [{
"finish_reason": "tool_calls",
"index": 0,
"message": {
"role": "assistant",
"tool_calls": [{
"extra_content": {"google": {"thought_signature": "Ep0CCpoC…(truncated)"}},
"function": {"arguments": "{\"city\":\"Paris\"}", "name": "get_weather"},
"id": "call_482567",
"type": "function"
}]
}
}],
"created": 1786935455,
"model": "gemini-3.7-flash",
"object": "chat.completion",
"usage": {"completion_tokens": 16, "prompt_tokens": 55, "total_tokens": 117}
}
```

**Streaming** — same payload plus `"stream": true, "stream_options": {"include_usage": true}`:

```bash
curl -s https://generativelanguage.googleapis.com/v1beta/openai/chat/completions \
-H "Authorization: Bearer $GEMINI_API_KEY" \
-H "Content-Type: application/json" \
-d @payload_stream.json
```

Full SSE output (HTTP 200, `id: oXiCaoOBBsPg8QGa35aRCw`, signatures truncated):

```
data: {"choices":[{"delta":{"role":"assistant","tool_calls":[{"extra_content":{"google":{"thought_signature":"EpwCCpkC…(truncated)"}},"function":{"arguments":"{\"city\":\"Paris\"}","name":"get_weather"},"id":"call_1839541","type":"function"}]},"index":0}],"created":1786935457,"id":"oXiCaoOBBsPg8QGa35aRCw","model":"gemini-3.7-flash","object":"chat.completion.chunk","usage":{...}}

data: {"choices":[{"delta":{"role":"assistant"},"finish_reason":"stop","index":0}],"created":1786935457,"id":"oXiCaoOBBsPg8QGa35aRCw","model":"gemini-3.7-flash","object":"chat.completion.chunk","usage":{...}}

data: [DONE]
```

The tool call arrives complete (id + name + full arguments) in a single delta chunk; the terminal chunk then reports `finish_reason: "stop"` even though the only content produced was a tool call. Expected: `"tool_calls"`, matching the non-streaming response for the same request.

## Impact / ecosystem context

Because the non-streaming path already returns `"tool_calls"`, this looks like a bug in the streaming path of the compat layer rather than an intentional mapping. Downstream, several projects now carry independent normalization patches for it, e.g. [BerriAI/litellm#21041](https://github.com/BerriAI/litellm/issues/21041), [BerriAI/litellm#12249](https://github.com/BerriAI/litellm/issues/12249), and agent-side breakage reports like [anomalyco/opencode#14972](https://github.com/anomalyco/opencode/issues/14972); OpenRouter also rewrites the value at their edge.

We run an OpenAI-compatible gateway (clawapi.org) that routes to Gemini among other upstreams; we have already shipped the same normalization on our side (stream-side `stop` → `tool_calls` when tool-call deltas were emitted), so this report is not blocking us — filing it so the fix can land at the source and the ecosystem can eventually drop these patches.

## Related observation (separate, minor)

In the same streaming capture, entries inside the `tool_calls` delta array carry no `index` field (OpenAI clients use it to assemble parallel tool calls; some SDK codepaths `KeyError` on it). Previously reported on the forum in Jan 2025: [discuss.ai.google.dev/t/59886](https://discuss.ai.google.dev/t/gemini-openai-compatibility-issue-with-tool-call-streaming/59886). Mentioning for completeness since the capture above shows it; can file separately if useful.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.