Comfy-Org / Comfy-Org/ComfyUI

GeminiVideoOmni calls generateContent, but gemini-omni-flash-preview requires the Interactions API

Open
#14,961 3 comments 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

## Summary

The built-in **Google Gemini Omni (Video)** node fails on Comfy Cloud before generation starts.

```
API Error: gemini-omni-flash-preview is only supported in the Interactions API and cannot be called directly via generateContent.
```

The workflow inputs are valid: the prompt is connected through the prompt socket, three reference images are connected, and the selected model is **Omni Flash**.

## Steps to reproduce

1. Open Comfy Cloud.
2. Add the built-in `Google Gemini Omni (Video)` / `GeminiVideoOmni` node.
3. Select `Omni Flash`.
4. Connect a non-empty prompt (reference images are optional; the reported case uses three).
5. Run the workflow.

## Actual result

The node immediately fails with the API error above. The node reports 0 seconds of server processing, so the request is rejected before video generation begins.

## Expected result

The node should call the Gemini **Interactions API** and return the generated video/audio plus any text response.

Google's Gemini Omni Flash documentation uses:

- SDK: `client.interactions.create(model="gemini-omni-flash-preview", ...)`
- REST: `POST https://generativelanguage.googleapis.com/v1beta/interactions`

Documentation: https://ai.google.dev/gemini-api/docs/omni

## Source-level root cause

In `comfy_api_nodes/nodes_gemini.py`, `GeminiVideoOmni.execute` currently submits:

```python
ApiEndpoint(path=f"{GEMINI_BASE_ENDPOINT}/{model_id}", method="POST")
```

with a `GeminiGenerateContentRequest` and `GeminiGenerateContentResponse`. That is the existing generateContent path used by other Gemini nodes, but `gemini-omni-flash-preview` explicitly rejects generateContent.

The node was introduced in #14695. Its PR checklist shows QA was not completed.

## Suggested fix

1. Add/use a Comfy proxy route for Gemini `v1beta/interactions`.
2. Serialize prompt/images/videos into the Interactions `input` schema.
3. Parse video output from the Interactions response `steps` array (or the SDK convenience `output_video` field where applicable).
4. Preserve current billing extraction and video download behavior.
5. Add a regression test asserting that `GeminiVideoOmni` does not use the generateContent endpoint/request model.

## Environment

- Product: Comfy Cloud
- Node: `GeminiVideoOmni`
- Model: `gemini-omni-flash-preview`
- Observed: 2026-07-17

Contributor guide

Open the contributing guide

Research direction

Start in comfy_api_nodes/nodes_gemini.py at GeminiVideoOmni.execute and compare its current generateContent request with the Gemini Interactions API requirements. Trace the Comfy proxy route, input serialization, response steps, billing extraction, and video download behavior. Add a regression test confirming the node no longer uses the generateContent endpoint/request model and verify video, audio, and text outputs remain available.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.