google-gemini / google-gemini/gemini-cli
bug: voice providers resolve connect before transcription is ready
- Dominant language
- TypeScript
- Stars
- 107k
- Forks
- 14.6k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 45
Description
## What happened?
The voice transcription providers can resolve `connect()` before they are usable, causing callers to start recording and send audio to a dead or incompletely initialized provider.
Two implementations exhibit the same contract problem:
### Local Whisper
In `packages/core/src/voice/whisperTranscriptionProvider.ts:70-151`, readiness normally waits for the `main: processing` stderr marker. However:
- The child `close` handler does not reject a still-pending connection.
- A ten-second fallback resolves unconditionally if readiness was never observed.
- The timeout is not cleared after success, error, or close.
If `whisper-stream` exits before becoming ready without emitting an `error` event or an stderr line containing lowercase `error`, `connect()` still resolves ten seconds later even though `this.process` is already null.
### Gemini Live
In `packages/core/src/voice/geminiLiveTranscriptionProvider.ts:66-144`, `connect()` waits only for the WebSocket `open` event, sends the setup message, and returns immediately. It does not wait for the protocol's setup-complete response. The message handler processes only `serverContent`, so setup acknowledgement/rejection is not part of connection readiness.
## How can this be reproduced?
- Whisper: mock the spawned child to emit `close` before the readiness marker, advance timers ten seconds, and observe `connect()` resolve.
- Gemini Live: mock a socket that emits `open` but never sends setup completion, and observe `connect()` resolve immediately after `send()`.
The voice hook starts the recorder immediately after awaiting `connect()`, making this externally visible as lost initial audio or a connected UI backed by no live provider.
## What did you expect to happen?
`connect()` should resolve only when the provider is ready to accept audio. It should reject on close/error/setup rejection before readiness, and all readiness timers/listeners should be cleaned up on every terminal path.
## Why this matters
The current behavior produces silent voice failures and misleading connected state, which is especially confusing because the UI has already completed model/provider setup.
## Suggested direction
- Define readiness semantics explicitly in the `TranscriptionProvider` contract.
- Reject Whisper connection on pre-ready close and make timeout a rejection, not success.
- Wait for Gemini Live setup completion before resolving.
- Add provider lifecycle tests covering ready, early close, error, and timeout.
## Client information
Found by source audit on current `main`, commit `f47d6c6f7`. Whisper currently has only an unavailable-binary test; Gemini Live has no dedicated provider test.
Contributor guide
Research direction
Read packages/core/src/voice/whisperTranscriptionProvider.ts:70-151 and packages/core/src/voice/geminiLiveTranscriptionProvider.ts:66-144, then inspect the TranscriptionProvider contract and existing Whisper unavailable-binary test. Add lifecycle coverage for ready, early close, error, setup rejection, and timeout paths; done means connect() resolves only after readiness and rejects or cleans up on every failed terminal path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- audio-video-rtc, backend-api-design
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100