googleapis / googleapis/python-genai
Add `speech_started` and `speech_stopped` events for automatic VAD in Gemini Live
- Lingua principale
- Python
- Stelle
- 4k
- Fork
- 1k
- Merge medio
- 2g 12h
- PR unite (30g)
- 41
Descrizione
**The problem**
When using Gemini Live with **automatic voice activity detection (VAD)**, there is no way for the client to know when the model detects the *start* or *end* of user speech.
OpenAI’s Realtime API provides two very useful events for this:
* `input_audio_buffer.speech_started` → emitted as soon as the model’s VAD detects that the user has begun speaking.
* `input_audio_buffer.speech_stopped` → emitted as soon as the model’s VAD detects that the user has stopped speaking.
These signals are important for building real-time experiences (e.g., showing microphone indicators, animating avatars, syncing captions, or debugging latency).
Gemini Live does not expose such events today. The only related flag is `interrupted`, but that serves a different purpose (signaling that the *model’s response* was cut off by user speech, not speech boundary detection).
---
**The solution**
Please add support for Gemini Live responses to include explicit events or fields for:
* `speech_started` — when automatic VAD detects user speech onset.
* `speech_stopped` — when automatic VAD detects user speech offset.
This would mirror OpenAI Realtime’s design and make it much easier to integrate Gemini Live into responsive UIs without requiring developers to implement their own manual VAD logic.
---
**Alternatives:**
* Using manual VAD mode (`activityStart` / `activityEnd`) and sending these signals ourselves. However, this defeats the purpose of using automatic VAD, since it requires implementing and maintaining our own VAD pipeline.
* Inferring speech start/stop indirectly (e.g., based on when Gemini responds), which is unreliable and adds latency.
Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.