google-gemini / google-gemini/live-api-web-console

Gemini does not detect video or voice.

Open
#108 4 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
TypeScript
Stars
2.6k
Forks
731
PR merge metrics
No merged PRs in 30d

Description

### Description of the bug:

Hi there!
When running this app, whenever I turn on my mic and my webcam, it doesn't respond anything to me, I checked the logs, and see that it's repeatedly sending video data and preventing from the actual API to process

Here's the log in the app provides

8:26:51client.openConnected
8:26:51server.sendsetupComplete
8:26:55client.realtimeInputaudio39
8:26:55client.realtimeInputvideo
8:26:57client.realtimeInputaudio31
8:26:57client.realtimeInputvideo
8:26:59client.realtimeInputaudio29
8:26:59client.realtimeInputvideo
8:27:07client.realtimeInputaudio248
8:27:07client.realtimeInputvideo
8:27:08client.realtimeInputaudio45

When turning off the webcam and the mic, and only using pure text, it responds normally, but seems to hallucinate and tells me that it hears me and sees my video feed???

Secondly, when using the direct SDK provided by Google Studio AI, sometimes it seems to also hallucinate and pretends that it doesn't hear me and doesn't see my webcam (even though I am speaking to it and streaming via OpenCV) (PS The second issue, I used the latest 2.5 native thinking audio model, when I switch back to the older 2.0 Version, it stops hallucinating. And yes, I tried changing API keys and restarting to try to "reset" the chatlogs (Or tried to since it still remember what I have said earlier???)

### Actual vs expected behavior:

_No response_

### Any other information you'd like to share?

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.