Now that Gemini Flash can output images in amongst text, need to figure out how to model that
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 12.5k
- Forks
- 991
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 10
Description
See https://simonwillison.net/2025/May/7/gemini-images-preview/ - it's now possible to get a response back from gemini-2.0-flash-preview-image-generation which includes inline images like this:
{
"candidates": [
{
"content": {
"parts": [
{
"text": "## Vegetarian \"Butter Chicken\" with Chickpeas\n\nThis recipe delivers the rich, creamy flavors of classic butter chicken, substituting tender chickpeas for chicken. It's a satisfying and flavorful vegetarian main course perfect for any night of the week.\n\n**Yields:** 4-6 servings\n**Prep time:** 20 minutes\n**Cook time:** 35 minutes\n\n**Ingredients:**\n\n* **For the Chickpeas:**\n * 2 (15-ounce) cans chickpeas, drained and rinsed\n "
},
{
"inlineData": {
"mimeType": "image/png",
"data": "iVBORw0KGgoAAAANSUhEUgAAAtgAAAQAC..."
}
},
{
"text": "\n * 1 tablespoon olive oil\n "
},
{
"inlineData": {
"mimeType": "image/png",
"data": "iVBORw0KGgoAAAANSUhE..."
}
},
{
"text": "\n * 1 teaspoon garam masala\n "
},
{
"inlineData": {
"mimeType": "image/png",
"data": "iVBORw0KGgoAAAANS..."
}
}
]
}
}
]
}
Full example here: https://gist.github.com/simonw/55894032b2c60b35f320b6a166ded493
I think we need to store these in the DB, even though they can be really big (that single response was 41MB!)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No repository file or test is named; start with the linked Gemini response example and inspect the existing response-storage entry point. Decide how mixed text and inline images should be represented and persisted, including the 41MB response case; done means the model and storage behavior are defined and covered by tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, backend, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100