googleapis / googleapis/python-genai
Auto-injected 'propertyOrdering' makes gemini-2.5-pro emit near-empty JSON on long-form extraction (finishReason=STOP, no error)
- Dominant language
- Python
- Stars
- 4k
- Forks
- 1k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 40
Description
Is this a client library issue or a product issue? **Library**.
tl;dr: The SDK auto-injects `propertyOrdering` into the request, and `gemini-2.5-pro` reacts to that property by silently dropping output fields ~35–65% of the time on long-form prose inputs.
#### Environment details
- Programming language: Python
- OS: macOS 25.4 / Linux (Cloud Run)
- Language runtime version: 3.11 / 3.12
- Package version: `google-genai==1.33` (also reproduces on 1.31)
#### Issue
1. **`google-genai` injects `propertyOrdering` into the response schema by default.** When you pass a Pydantic `BaseModel` as `response_schema`, `_transformers.process_schema(..., order_properties=True)` adds `propertyOrdering` at every object level. You can't disable it without bypassing the SDK.
2. **gemini-2.5-pro still reasons correctly about every field.** Set `include_thoughts=True` and you can see the model's reasoning trace listing the values for each field of the response schema: skills, location, languages, etc.
3. **But 20–30% of the time on long-form prose extraction, the model emits a near-empty JSON anyway.** Same prompt, same schema, same input, temperature 0, the same query returns a fully populated response on one call and a near-empty one on the next. `finishReason=STOP`, JSON parses cleanly, no client-side error, no truncation, no safety filter, only the thinking trace reveals the dropped fields.
Isolated by sending the same prompt + input through Vertex REST 5 ways, varying only the JSON schema (n=20 per variant):
| schema variant | near-empty emission rate |
|---|---|
| baseline (no `propertyOrdering`) | 0% |
| baseline + uppercase types | 0% |
| baseline + `nullable: true` | 0% |
| baseline + **`propertyOrdering` only** | **65%** (13/20) |
Unaffected on the same prompt + input: `gemini-2.5-flash` with `thinking_budget=-1` → 0/20;
#### Steps to reproduce
1. `pip install httpx pydantic google-auth`
2. Run `minimal_repro.py` (self-contained, uses `pydantic` for the schema, no `google-genai` needed): `gcloud auth application-default login && GOOGLE_CLOUD_PROJECT= python minimal_repro.py --n 20`
3. The script sends `n` calls each through two variants of the SAME Pydantic-derived JSON schema, on the SAME prompt and the SAME public Dutch JD:
- **A**: schema without `propertyOrdering` (clean `model_json_schema()` output with refs inlined)
- **B**: schema with `propertyOrdering` injected at every object level (what `google-genai._transformers.process_schema` does automatically)
4. The schema has 6 user-facing fields the model should fill from the JD: `must_have_titles`, `must_have_skills`, `nice_to_have_skills`, `location`, `languages`, `preferred_industries`. We measure **how many of those 6 ended up populated** in each response.
5. Observed at n=40 per variant on the fictional Dutch JD:
| Variant | Rich output (≥4 of 6 populated) | Near-empty (0 or 1 of 6 populated) |
|---|---|---|
| A no `propertyOrdering` | **38/40 (95%)** | **2/40 (5%)** |
| B with `propertyOrdering` | **27/40 (68%)** | **12/40 (30%)** including **4 totally empty** responses |
With `propertyOrdering` injected the model returns near-empty output **6× more often**. On those failing responses, every Optional field except (often) `must_have_titles` is silently defaulted to `null` / `[]`, `finishReason=STOP`, no error.
6. A richer internal Pydantic class (more nested fields, three-way industry split, additional `Literal` enums) raises the variant-B near-empty rate to **35–65%**; the structural pattern that triggers the bug is the same.
The bug reproduces when ALL of these conditions hold:
- gemini-2.5-pro (NOT Flash, Flash with `thinking_budget=-1` is 0% on the same setup)
- Response schema with ~10+ Optional fields, nested object blocks, and lists of structured items (Pydantic models with `Optional[NestedClass] = None` and `tuple[Ref, ...] = ()` defaults)
- Long prose input (~4000+ chars, esp. Dutch)
- Long "conservative-extraction" system prompt (~10k+ chars, repeated "leave defaults" rules)
- `propertyOrdering` injected on the response schema (the SDK does this by default)
[minimal_repro.py](https://github.com/user-attachments/files/27656631/minimal_repro.py)
Contributor guide
Assessment
This issue has not been assessed yet.