MemberJunction / MemberJunction/MJ
Gemini: valid JSON body followed by stray trailing fragment despite ResponseFormat=JSON — runner should recover strict JSON centrally
- Dominant language
- TSQL
- Stars
- 29
- Forks
- 6
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 323
Description
## Summary
With a prompt configured `ResponseFormat=JSON` (Gemini driver sets `responseMimeType: application/json`), Gemini 3.1 Pro intermittently emits a **valid JSON object followed by a stray trailing fragment** — e.g. a duplicated closing `."\n}` after the balanced body. `JSON.parse` of the full `rawResult` then fails ("Unexpected non-whitespace character after JSON"), and a perfectly good result gets discarded by the consumer.
Observed in production 2026-07-06 on an audio-evaluation prompt (single-model path, Gemini 3.1 Pro via the Google vendor). Low frequency but recurring.
## Why this belongs in MJ rather than every consumer
`ResponseFormat=JSON` is a prompt-level contract. Today each consumer that parses `rawResult` has to reimplement recovery (or lose valid results to their retry path). If the runner (or the Gemini driver) validated the output when `ResponseFormat=JSON` and recovered the leading balanced JSON object, every consumer would benefit and the contract would actually hold.
## Suggested fix
When `ResponseFormat=JSON`:
1. Attempt strict `JSON.parse` of the model output.
2. On failure, extract the first balanced top-level JSON object with a **string-aware** brace scan (braces/quotes inside string literals, including escaped quotes, must not affect depth) and re-parse.
3. If recovery succeeds, log a warning with the discarded tail; if not, fail as today. Deliberately do **not** attempt to repair truncated/unbalanced JSON — a truncated body should still be an error.
We wrote exactly this as a downstream workaround (`extractFirstJsonObject`, ~50 lines + unit tests covering the observed prod shape, escaped quotes, nesting, and null cases) and are happy to contribute it upstream if there's interest.
## Version
`@memberjunction/ai-prompts` / `@memberjunction/ai-gemini` 5.44.0
Contributor guide
Research direction
Start by tracing where the runner or Gemini driver handles output when ResponseFormat=JSON, then review the downstream extractFirstJsonObject workaround and its unit-test cases. Done means valid leading balanced JSON is recovered without accepting truncated bodies, discarded trailing text is warned about, and unrecoverable output still fails as before.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- json
- Domain
- ai, backend-api-design, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 65/100