GoogleCloudPlatform / GoogleCloudPlatform/generative-ai
[Bug]: Overlapping Audio Causes Misinterpreted User Input in WebSocket Demo App
- Dominant language
- Jupyter Notebook
- Stars
- 17.7k
- Forks
- 4.5k
- Avg merge
- 12h 38m
- Merged PRs (30d)
- 42
Description
### File Name
gemini/multimodal-live-api/websocket-demo-app
### What happened?
When using the WebSocket Demo App in the Gemini Multimodal Live API repository, there is an issue where audio inputs overlap during user speech. This overlapping causes the app to mistakenly treat parts of the audio response as if they were user responses, thereby disrupting the natural conversation flow.
**Steps to Reproduce:**
1. Launch the WebSocket Demo App from the [generative-ai repository](https://github.com/GoogleCloudPlatform/generative-ai/tree/main/gemini/multimodal-live-api/websocket-demo-app).
2. Start a conversation by speaking into the microphone.
3. During the conversation, observe that when the user is speaking, overlapping audio (from system responses) is captured concurrently.
4. Notice that audio segments received during the overlap are interpreted as user input, which causes the conversation to continue erroneously.
Please let me know if further details are needed to help diagnose this issue.
### Code of Conduct
- [x] I agree to follow this project's Code of Conduct
Contributor guide
Assessment
This issue has not been assessed yet.