GoogleCloudPlatform / GoogleCloudPlatform/gcc-creative-studio
Gemini Omni video generation ignores several documented request parameters and lacks multi-turn editing
- Dominant language
- Python
- Stars
- 144
- Forks
- 385
- Avg merge
- 1d 46m
- Merged PRs (30d)
- 5
Description
While extending Creative Studio's Gemini Omni video support, I found the app diverges from what the [Interactions API](https://docs.cloud.google.com/gemini-enterprise-agent-platform/reference/models/interactions-api) actually supports (Google Cloud reference; see also the [AI Studio docs](https://ai.google.dev/gemini-api/docs/omni)), in ways that degrade generation quality without any visible error:
- **Aspect ratio was silently ignored.** Requests for 9:16 returned 16:9 regardless.
- **Duration was silently discarded**, and separately bounded to values Omni doesn't actually use.
- **Multi-turn video editing (`task=edit`) wasn't implemented at all.** The app could generate new clips but had no way to modify an existing one — the documented Vertex pattern of sending `[text, image(s), video]` with `task=edit` was unused.
- **Role tags (``) worked at the API level but weren't reliably wired up in the app.** In one code path, reference images were attached in reversed order, so `` silently bound to the wrong image — confirmed via a live-API test with 3 references and a positional swap.
- **Reference video input in "Ingredients to Video" was accepted by the schema but not actually usable** — the correct pattern for combining a video with images is an edit, not a reference.
- **End-frame / video-extension requests against Omni return unclear failures.** Omni cannot do first+last-frame interpolation or true video extension, but the app didn't distinguish this from other models.
- **Edit requests are rejected with a misleading error** (`This model does not support video extension`) depending on prompt wording alone, even when the request correctly specifies `task=edit`. Isolated via 116 live API calls varying one variable at a time — full writeup in the PR.
- **Audio reference input is exposed in the UI but rejected outright by the API** for this model.
Contributor guide
Research direction
Start with the Creative Studio code that handles Gemini Omni video requests and compare its behavior with the linked Interactions API and AI Studio documentation. Trace parameter handling, edit requests, reference ordering, and model-specific validation. Done means supported parameters and multi-turn editing work correctly, unsupported inputs are clearly rejected, and misleading errors are distinguished.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, python
- Domain
- ai, api
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100