googleapis / googleapis/python-genai
[Bug] JSONDecodeError "Extra data" when parsing streamed raw JSON lines (Agent Engine stream_query)
- Dominant language
- Python
- Stars
- 4k
- Forks
- 1k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 40
Description
## Summary
`google-genai` fails to parse streaming responses from Vertex AI Agent Engine (`stream_query`) with `JSONDecodeError: Extra data`. The raw response shows multiple JSON objects concatenated.
## Environment
- google-genai: 1.62.0
- google-adk: 1.22.0
- google-cloud-aiplatform: 1.136.0
- httpx: 0.28.1
- Python: 3.13.12
- Backend: Vertex AI Agent Engine
- API: `stream_query` with `?alt=sse`
- Observed Content-Type: `application/json`
## Call Chain
`google-cloud-aiplatform` (vertexai) uses `google-genai` as its internal HTTP client. The error occurs inside `google-genai` when parsing the streamed response:
```
app code
→ vertexai.AgentEngine.stream_query() # google-cloud-aiplatform
→ vertexai._genai.agent_engines._stream_query() # google-cloud-aiplatform
→ google.genai._api_client.request_streamed() # google-genai
→ HttpResponse.segments() # google-genai
→ HttpResponse._iter_response_stream() ← brace-count buffering bug
→ HttpResponse._load_json_from_response() ← JSONDecodeError raised here
```
## Observed Request/Response (sanitized app-side logs)
Source: application-side logs (not genai internal logs).
Interpretation: Although `?alt=sse` is requested, the backend responds with `Content-Type: application/json` and chunked transfer. The raw response is not SSE `data:` lines but plain JSON lines.
```
[INFO] response_content_type=application/json
[INFO] response_headers={'content-type': 'application/json', 'transfer-encoding': 'chunked', 'date': 'REDACTED', 'server': 'scaffolding on HTTPServer2'}
[INFO] request method=POST url=https://redacted-aiplatform.googleapis.com/v1beta1/projects/REDACTED/locations/REDACTED/reasoningEngines/REDACTED:streamQuery?alt=sse accept=*/*
```
## Evidence of concatenated JSON (sanitized app-side logs)
Source: application-side logs (not genai internal logs).
These logs are from read-only instrumentation (a wrapper around `HttpResponse._iter_response_stream` that only observes and logs without modifying behavior) that records each line's brace delta (`{` count minus `}` count) and running balance. `concat=True` means the line contains a `}{` boundary indicating concatenated JSON objects.
Interpretation: The first raw JSON line is large and has `delta=+1`, so the brace counter never returns to zero. The next JSON event is appended to the same buffer, producing a `}{` boundary and `json.loads` fails with `Extra data`.
```
[WARNING] raw_line len=143547 bytes=143547 concat=True delta=1 balance=1 sample={"content": {"parts": [{"function_response": ...}] ...}}
[ERROR] json_decode_error response_len=152621 response_bytes=152621 concat=True concat_ctx=[REDACTED]}{[REDACTED]
lines=idx=1 kind=raw len=143547 bytes=143547 concat=True delta=1 balance=1 ...
| idx=2 kind=raw len=9074 bytes=9074 concat=False delta=0 balance=1 ...
```
## Error Message (actual)
```
google.genai.errors.UnknownApiResponseError: Failed to parse response as JSON.
Raw response: {"content": {"parts": [{"function_response": ...}] ... }{"model_version": "gemini-2.5-pro", ...}
```
## Stack Trace (excerpt)
```
... _api_client.py:1404 request_streamed
... _api_client.py:284 segments
... _api_client.py:447 _load_json_from_response
json.decoder.JSONDecodeError: Extra data
```
## Reproduction Steps
1. Deploy an Agent Engine with tools that return large responses containing `{`/`}` in text (e.g., internal wiki pages or documents with logs/JSON snippets).
2. Invoke `stream_query()`.
3. When tool output is large, the response is `Content-Type: application/json` and appears as raw JSON lines.
4. Parsing fails with `JSONDecodeError: Extra data`.
## Minimal Repro (standalone)
This reproduces the same failure locally without Vertex AI by feeding raw JSON lines into `google.genai._api_client.HttpResponse`.
Although it uses internal classes, it exercises the same path used by `request_streamed() -> HttpResponse.segments()`,
which is where `stream_query()` fails in production when the backend returns raw JSON lines with `Content-Type: application/json`.
```python
#!/usr/bin/env python3
"""Minimal local reproduction using google-genai internals.
This feeds raw (non "data: ") lines into google.genai._api_client.HttpResponse
so its brace-count buffering is exercised. Braces inside JSON strings unbalance
the counter, leading to concatenated JSON and a json.loads "Extra data" error.
"""
from __future__ import annotations
import json
import httpx
from google.genai import _api_client as genai_api_client
from google.genai import errors
def main() -> None:
# Line1 simulates a large tool response JSON that contains an extra "{"
# inside a string. This unbalances the brace counter.
line1 = json.dumps(
{
"content": {
"parts": [
{
"function_response": {
"id": "tool-1",
"name": "get_document",
"response": {
"id": "12345",
"title": "example",
"content": {
"value": "log line with unmatched brace {",
"format": "markdown",
},
},
}
}
]
},
"id": "evt1",
}
)
# Line2 simulates the next JSON event.
line2 = json.dumps(
{
"model_version": "gemini-2.5-pro",
"content": {"parts": [{"text": "summary"}]},
"id": "evt2",
}
)
lines = [line1, line2]
content = ("\n".join(lines)).encode("utf-8")
response = httpx.Response(200, content=content)
http_response = genai_api_client.HttpResponse(response_stream=response, headers={})
print("Raw line lengths:", [len(line) for line in lines])
print("Brace deltas:", [line.count("{") - line.count("}") for line in lines])
print("Attempting to parse via google.genai._api_client.HttpResponse.segments()")
try:
# This should raise UnknownApiResponseError due to concatenated JSON.
list(http_response.segments())
except errors.UnknownApiResponseError as exc:
print("Caught UnknownApiResponseError")
print(f"Message: {exc}")
if exc.__cause__:
print(f"Cause: {exc.__cause__}")
else:
print("No error raised (unexpected).")
if __name__ == "__main__":
main()
```
Install:
```
pip install google-genai==1.62.0 httpx==0.28.1
```
Run:
```
python repro_genai_json_decode_error.py
```
Output:
```
Raw line lengths: [243, 94]
Brace deltas: [1, 0]
Attempting to parse via google.genai._api_client.HttpResponse.segments()
Caught UnknownApiResponseError
Message: Failed to parse response as JSON. Raw response: {"content": {"parts": [{"function_response": {"id": "tool-1", "name": "confluence_get_page", "response": {"id": "2982183028", "title": "example", "content": {"value": "log line with unmatched brace {", "format": "markdown"}}}}]}, "id": "evt1"}{"model_version": "gemini-2.5-pro", "content": {"parts": [{"text": "summary"}]}, "id": "evt2"}
Cause: Extra data: line 1 column 244 (char 243)
```
## Actual Behavior
`google-genai` raises `UnknownApiResponseError` due to `JSONDecodeError: Extra data` when multiple JSON objects appear concatenated in the raw response.
## Expected Behavior
Streaming responses should be parsed successfully, or concatenated JSON objects should be split safely.
Contributor guide
Assessment
This issue has not been assessed yet.