MemberJunction / MemberJunction/MJ

Gemini: valid JSON body followed by stray trailing fragment despite ResponseFormat=JSON — runner should recover strict JSON centrally

Open
#3,065 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TSQL
Stars
29
Forks
6
Avg merge
2d 1h
Merged PRs (30d)
323

Description

## Summary

With a prompt configured `ResponseFormat=JSON` (Gemini driver sets `responseMimeType: application/json`), Gemini 3.1 Pro intermittently emits a **valid JSON object followed by a stray trailing fragment** — e.g. a duplicated closing `."\n}` after the balanced body. `JSON.parse` of the full `rawResult` then fails ("Unexpected non-whitespace character after JSON"), and a perfectly good result gets discarded by the consumer.

Observed in production 2026-07-06 on an audio-evaluation prompt (single-model path, Gemini 3.1 Pro via the Google vendor). Low frequency but recurring.

## Why this belongs in MJ rather than every consumer

`ResponseFormat=JSON` is a prompt-level contract. Today each consumer that parses `rawResult` has to reimplement recovery (or lose valid results to their retry path). If the runner (or the Gemini driver) validated the output when `ResponseFormat=JSON` and recovered the leading balanced JSON object, every consumer would benefit and the contract would actually hold.

## Suggested fix

When `ResponseFormat=JSON`:
1. Attempt strict `JSON.parse` of the model output.
2. On failure, extract the first balanced top-level JSON object with a **string-aware** brace scan (braces/quotes inside string literals, including escaped quotes, must not affect depth) and re-parse.
3. If recovery succeeds, log a warning with the discarded tail; if not, fail as today. Deliberately do **not** attempt to repair truncated/unbalanced JSON — a truncated body should still be an error.

We wrote exactly this as a downstream workaround (`extractFirstJsonObject`, ~50 lines + unit tests covering the observed prod shape, escaped quotes, nesting, and null cases) and are happy to contribute it upstream if there's interest.

## Version

`@memberjunction/ai-prompts` / `@memberjunction/ai-gemini` 5.44.0

Contributor guide

Open the contributing guide

Research direction

Start by tracing where the runner or Gemini driver handles output when ResponseFormat=JSON, then review the downstream extractFirstJsonObject workaround and its unit-test cases. Done means valid leading balanced JSON is recovered without accepting truncated bodies, discarded trailing text is warned about, and unrecoverable output still fails as before.

Written by the indexing model from the issue text.

Assessment

Tech stack
json
Domain
ai, backend-api-design, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.