googleapis / googleapis/python-genai

Feature: file uploading when using chat api

Open
#1,336 1 comment 0 reactions 1 assignee Claimed by @Venkaiahbabuneelam View on GitHub
priority: p3 type: feature request
Dominant language
Python
Stars
4k
Forks
1k
Avg merge
2d 11h
Merged PRs (30d)
40

Description

# Description of issue

In the deprecated generative-ai library, it was possible to send inline data alongside text messages as a _gemini's parts field list_ when using send_message. The format looked like this:
`[
{
"text": "",
"data_inline": {
"data": "",
"mime_type": ""
}
}
]
`

This was extremely useful for building a chatbot app where users can send both text and multiple file contents in the same message.

With the new genai library, this functionality no longer works and passing such a list to send_message now throws an error. Those errors suggest to use genai.types objects instead of lists, but i could not find such object that could help my case. However, interestingly, the parsing functionality (handling a list of mixed text and inline data objects inside one message's parts field and forwarding them properly to the service) still exists when passing history to the chat at its creation.

Implementing a custom send_message is possible but it feels like re-inventing functionality that already exists internally.

# Workaround

I patched this by modifying send_message to use the existing extraction/parsing functionality from history and then append it to the chat history. It works, but it feels hacky and obviously not efficient and probably minimizes the abstract nature of the function:

**pseudo-code of workaround**
```
async def send_message(message):
......
self.history.extend(message) #self.history contains the history in JSON as it was given while initializing the chat
data_to_send=_extract_curated_history(history)
response = generate_content(
contents=data_to_send
)
.....
```
Note that for this to work message must have this format
```
[
{"role": "user,"
parts": [{"text":.... ,etc.}]
}
```

# Request

Please consider adding support for inline data in send_message directly by passing the parts list. This would make migration from generative-ai smoother and support richer chatbot use cases out of the box.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.