google-gemini / google-gemini/gemini-live-api-examples

`gemini-3.1-flash-live-preview`: the model repeats the same sentence 2–4 times inside one generation (~7 s of silence between each) before releasing the toolCall

Open
#46 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
496
Forks
175
Avg merge
6m
Merged PRs (30d)
1

Description

### Description of the bug

On `gemini-3.1-flash-live-preview` (Live API over API key, native audio, `thinkingConfig.thinkingLevel = LOW`, server-side VAD), when the system instruction asks the model to confirm a piece of data and then persist it with a function call, the model says the confirmation once and then **repeats the same sentence verbatim 2–4 times inside a single generation**, and only then releases the `toolCall`. Each repetition is preceded by **~7 s of complete server silence** (no message of any kind on the socket), then an output transcription that literally reads `\n\n(in attesa) ...` ("(waiting) ..."), then the sentence again. No `interrupted`, no `turnComplete`, no new generation in between: it is one generation stalling and re-emitting itself until the function call comes out, 10–30 s after the first confirmation. The caller hears the agent say "Perfetto, useremo l'arabo per la mediazione. Un attimo che aggiorno il sistema." four times in a row.

Raw server messages of one generation (full stack: `livekit-plugins-google` 1.8.1 on top of `google-genai` 2.22.0, production system instruction of 32,220 characters, session opened `2026-09-15T18:43:46Z`, times in seconds since session start; complete excerpt in the gist):

```
36.03 generation starts | inputTranscription: 'Buongiorno, ho bisogno di una mediazione in arabo.'
38.23 outputTranscription: 'Perfetto, useremo'
38.59 outputTranscription: " l'ARABO" ... ' per la' ... ' mediazione.'
38.97 outputTranscription: ' Un attimo' ... ' soltanto.' <- confirmation #1 complete
(7.2 s: no server message at all)
46.57 outputTranscription: 'Perfetto, useremo' ... " l'arabo per la mediazione."
47.33 outputTranscription: ' Un attimo' ... ' che aggiorno' ... ' il sistema.' <- #2
(7.1 s: no server message at all)
54.81 outputTranscription: '\n\n(in attesa) ...'
54.98 outputTranscription: '\nPerfetto, useremo' ... " l'arabo per la mediazione."
55.76 outputTranscription: ' Un attimo' ... ' che aggiorno' ... ' il sistema.' <- #3
(6.6 s: no server message at all)
62.64 outputTranscription: '\n\n(in attesa) ...'
62.80 outputTranscription: '\nPerfetto, useremo' ... " l'arabo per la mediazione."
63.56 outputTranscription: ' Un attimo' ... ' che aggiorno' ... ' il sistema.' <- #4
63.94 outputTranscription: '\n\n(in attesa) ...'
64.00 toolCall update_call_info {"call_info": {"lingua_richiesta": "ARABO"}} (18.35 s of audio, 28 s after #1)
```

Same signature with the pure SDK, no framework, one caller utterance, one 4.4k-character generic system instruction (`repro_live_repeat.py --system full`, run 03 in the gist):

```
16.57 caller audio ends (2.9 s, 16 kHz PCM): "Buongiorno, ho bisogno di una mediazione in cinese."
17.37 inputTranscription: "Buongiorno, ho bisogno di una mediazione in cinese."
17.99 outputTranscription: "Perfetto, useremo" ... " mediazione."
18.74 outputTranscription: " Un attimo." <- confirmation #1 complete
20.87 outputTranscription: "Perfetto, useremo" <- verbatim repeat, same generation
21.35 outputTranscription: " mediazione."
21.60 outputTranscription: " Un secondo."
21.79 toolCall update_call_info {"language":"Cinese"}
22.15 generationComplete (no interrupted, no turnComplete in between)
```

### How often, and what it depends on

| setup (all `gemini-3.1-flash-live-preview`, `thinkingLevel=LOW`, same VAD config) | trials | repeated ≥2× | notes |
|---|---|---:|---|
| full stack, production instruction (32k chars), script says "save the language immediately" | 10 | **5** | up to 4×, 16.9–24.0 s of audio before the call; UTC session starts of the 5 affected trials: 18:43:46, 18:45:41, 18:49:03, 18:50:30, 18:52:37 (2026-09-15) |
| same, script changed to "save only at the end of the call" (no mid-call function call expected) | 10 | 1 | the one residual case has no tool call in the turn |
| pure SDK, generic 4.4k instruction, 3 tools | 8 | 1 | run 03 above |
| pure SDK, 0.3k instruction, 1 tool | 8 | 0 | |
| pure SDK, generic 27k instruction, 3 tools | 20 | 0 | |

So in the pure SDK it is rare (1/36), in a full voice-agent stack with a production-size instruction it is 1 trial out of 2, and in our production traffic on this model it is **14 real calls out of 897** with agent transcripts (1.6%), 11 of the 21 repeated groups in a turn that ends with a function call (`hangup_call` 8, `update_lead_info` 3); the worst one repeats a whole question+answer block three times over 31 s before the call. We could not find what makes the full stack so much more prone than the bare script (the `LiveConnectConfig` is the same; the stack adds `sessionResumption`, a greeting turn generated from a client-content prompt, and continuous real-time caller audio).

### What we ruled out (150 live trials in the full stack, 15 per arm unless noted)

- **Not the instructions.** Six rewrites of the tool-use rules of the system instruction (say the waiting phrase once; no waiting phrase at all; a four-step "one sentence → tool → silence → continue" procedure with an explicit "never repeat a sentence"; "say nothing before the call") went from 5/10 to **12/15, 10/15, 14/15**: every rule that pushes the model towards the function call makes it repeat more, and the repetition is near-deterministic when the turn ends with the call (8/8, 5/5, 12/12). Removing the waiting phrase does not help: said in 2/15 trials, confirmation still repeated in 10/15. Only "emit the call without saying anything first" reduces it (3/15), and only because in 2/5 calls the model actually stayed silent — where it never repeats; it costs 7 empty generations and 4 unanswered caller turns.
- **It follows the intent to call a function**, not the words: dropping the mid-call save from the script (nothing else changed) goes 5/10 → 1/10.
- **`thinkingConfig` removed (`thinkingBudget: 0`, no `thinkingLevel`)** does not fix it, it removes the trigger: the model stops calling the function mid-conversation altogether (**0/15** where the script asks for it in five places; pure SDK: tool called in 1/4 runs vs 8/8 with `thinkingLevel=LOW`). 3/15 repetitions left, none with a tool call.
- **`FunctionDeclaration.behavior = NON_BLOCKING` + `scheduling = WHEN_IDLE`** (edit: this configuration is documented as *not supported* on `gemini-3.1-flash-live-preview` — "Asynchronous function calling is not yet supported in Gemini 3.1 Flash Live" — so the 13/15 we measured with it says nothing about the mechanism and is left out of the comparison).
- **`gemini-3.8-live` (released 2026-09-15; `thinkingLevel` is rejected by the server for it, so no `thinkingConfig`)**: 3/15, median audio before the call halves (7.8 s), but the repeated sentence becomes the waiting phrase itself ("Un secondo, aggiorno le informazioni" ×3, 10.2 s) — same mechanism. On 3.8 the shape is different: no silent gaps, the sentence is re-emitted back-to-back at ~1.3–1.5 s intervals, and it happens with `behavior: BLOCKING` set explicitly (2/10) as well as with `NON_BLOCKING` + `scheduling: WHEN_IDLE` (5/15, where the model also calls the function about twice as often); one 3.8 case repeats the confirmation twice with no function call in the turn at all. (Edit: the earlier remark about 3.8 never filling `call_info` was our own tooling — a string-argument workaround not applied to the new model — and has been removed.)

### Configuration

`LiveConnectConfig` as sent (key omitted): `response_modalities=[AUDIO]`; `generation_config.thinking_config={include_thoughts: false, thinking_level: LOW}`; `speech_config={voice: Charon, language_code: it-IT}`; `input_audio_transcription={}`, `output_audio_transcription={}`; `realtime_input_config={automatic_activity_detection: {start_of_speech_sensitivity: LOW, end_of_speech_sensitivity: LOW, prefix_padding_ms: 600, silence_duration_ms: 500}, activity_handling: START_OF_ACTIVITY_INTERRUPTS, turn_coverage: TURN_INCLUDES_ONLY_ACTIVITY}`; three blocking function declarations (`update_call_info`, `update_lead_info`, `hangup_call`). Caller audio: 16 kHz mono PCM, sent in real time in 20 ms chunks, then digital silence. `google-genai` 2.22.0, Python 3.13, macOS.

### Actual vs expected behavior

Expected: the confirmation is spoken once, then the `toolCall` is released (or the call comes first, as the model prefers). Actual: the same sentence is spoken 2–4 times inside one generation, separated by ~7 s of silence, and the `toolCall` arrives only at the end.

### What this report is not

It is not the "model speaks again after the tool response" duplicate of livekit/agents#4554, and it is not the missing function call of googleapis/python-genai#2827 — though both look like the same root: the sequencing between the thinking step, the audio stream and the release of the function call when `thinkingLevel` is set and tools are registered (with thinking off we get #2827's symptom instead: no call at all). It is not a VAD / self-interruption artefact: no `interrupted` event, caller silent, and it reproduces with the bare SDK.

### Reproduction

https://gist.github.com/IngLP/46fa251d46af9014fd6bca173128966a — `repro_live_repeat.py` is self-contained (only `GOOGLE_API_KEY`; it synthesises the caller audio with macOS `say`, `--audio` to supply your own), with `system_full.txt` / `system_full_long_v2.txt` (generic, no customer text), `repro_output.txt` (unedited output), `events_repeated_run.json` (the SDK run above) and `livekit_stack_trial04_server_events.txt` (the full-stack generation above, raw server messages).

```
python repro_live_repeat.py --system full --runs 8 # ~1 run in 8 shows the repetition
python repro_live_repeat.py --system full --runs 4 --no-thinking # tool call mostly disappears instead
```

Companion reports on this model from the same setup: #37, #38.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the linked gist and run `python repro_live_repeat.py --system full --runs 8` using the stated Gemini Live configuration. Compare the raw event logs with the expected single confirmation and prompt toolCall; done would mean identifying a reproducible SDK or API cause and preventing repeated audio before the call, though the report provides no project file to change.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, audio-video-rtc
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.