googleapis / googleapis/python-genai

google.genai only support batch embedding which leads to rate limit errors

Open
#427 2 comments 4 reactions 1 assignee Claimed by @jaycee-li View on GitHub
priority: p2 type: feature request
Dominant language
Python
Stars
4k
Forks
1k
Avg merge
2d 11h
Merged PRs (30d)
40

Description

Thanks for stopping by to let us know something could be better!

**PLEASE READ**: If you have a support contract with Google, please create an issue in the [support console](https://cloud.google.com/support/) instead of filing on GitHub. This will ensure a timely response.

Is this a client library issue or a product issue? We will only be able to assist with issues that pertain to the behaviors of this library. If the issue you're experiencing is due to the behavior of the product itself, please visit the [Support page](https://cloud.google.com/support) to reach the most relevant engineers.

If the support paths suggested above still do not result in a resolution, please provide the following details.

#### Environment details

- Programming language: Python
- OS: MacOS
- Language runtime version: 3.11
- Package version: google-genai==1.3.0

#### Steps to reproduce

1. Using a paid API
2. Run an embeddings job (sync)

```python

from tqdm import tqdm
import time

client = genai.Client(api_key=api_key)
error_counter = 0

for _ in tqdm(range(2000)):
success = False
while not success:
try:
response = client.models.embed_content(
model='text-embedding-004',
contents='Hello world',
)
success = True
except Exception as e:
print(e)
error_counter += 1
if "429" in (str(e)):
print("sleeping....")
time.sleep(10)

```

The function `embed_content` uses the following logic

```
path = '{model}:batchEmbedContents'.format_map(request_url_dict)
```

https://github.com/googleapis/python-genai/blob/91b1d3ee85fd32e9243e3a2da4d10a763e4d0005/google/genai/models.py#L4399

## The FIX

Please implement the following for users so that they stop hitting that 150 batch limit rate

```python

import requests
import json

def embed_content(api_key, text, model="text-embedding-004"):
"""Embeds text using the Gemini API.

Args:
api_key: Your Google Cloud API key.
text: The text to embed.
model: The embedding model to use.

Returns:
The JSON response from the API, or None if there was an error.
"""
url = f"https://generativelanguage.googleapis.com/v1beta/models/{model}:embedContent?key={api_key}"
headers = {"Content-Type": "application/json"}
data = {
"model": model,
"content": {
"parts": [{"text": text}]
}
}

try:
response = requests.post(url, headers=headers, data=json.dumps(data))
response.raise_for_status() # Raise an exception for error responses
return response.json()
except requests.exceptions.RequestException as e:
print(f"Error embedding content: {e}")
return None

# Example usage:

text = "Hello world"
response = embed_content(api_key, text)

if response:
print(response)

# test the code

for _ in tqdm(range(2000)):
embed_content(api_key, text)

```

https://ai.google.dev/api/embeddings#endpoint

Making sure to follow these steps will guarantee the quickest resolution possible.

Thanks!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.