googleapis / googleapis/python-genai
google.genai only support batch embedding which leads to rate limit errors
- Dominant language
- Python
- Stars
- 4k
- Forks
- 1k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 40
Description
Thanks for stopping by to let us know something could be better!
**PLEASE READ**: If you have a support contract with Google, please create an issue in the [support console](https://cloud.google.com/support/) instead of filing on GitHub. This will ensure a timely response.
Is this a client library issue or a product issue? We will only be able to assist with issues that pertain to the behaviors of this library. If the issue you're experiencing is due to the behavior of the product itself, please visit the [Support page](https://cloud.google.com/support) to reach the most relevant engineers.
If the support paths suggested above still do not result in a resolution, please provide the following details.
#### Environment details
- Programming language: Python
- OS: MacOS
- Language runtime version: 3.11
- Package version: google-genai==1.3.0
#### Steps to reproduce
1. Using a paid API
2. Run an embeddings job (sync)
```python
from tqdm import tqdm
import time
client = genai.Client(api_key=api_key)
error_counter = 0
for _ in tqdm(range(2000)):
success = False
while not success:
try:
response = client.models.embed_content(
model='text-embedding-004',
contents='Hello world',
)
success = True
except Exception as e:
print(e)
error_counter += 1
if "429" in (str(e)):
print("sleeping....")
time.sleep(10)
```
The function `embed_content` uses the following logic
```
path = '{model}:batchEmbedContents'.format_map(request_url_dict)
```
https://github.com/googleapis/python-genai/blob/91b1d3ee85fd32e9243e3a2da4d10a763e4d0005/google/genai/models.py#L4399
## The FIX
Please implement the following for users so that they stop hitting that 150 batch limit rate
```python
import requests
import json
def embed_content(api_key, text, model="text-embedding-004"):
"""Embeds text using the Gemini API.
Args:
api_key: Your Google Cloud API key.
text: The text to embed.
model: The embedding model to use.
Returns:
The JSON response from the API, or None if there was an error.
"""
url = f"https://generativelanguage.googleapis.com/v1beta/models/{model}:embedContent?key={api_key}"
headers = {"Content-Type": "application/json"}
data = {
"model": model,
"content": {
"parts": [{"text": text}]
}
}
try:
response = requests.post(url, headers=headers, data=json.dumps(data))
response.raise_for_status() # Raise an exception for error responses
return response.json()
except requests.exceptions.RequestException as e:
print(f"Error embedding content: {e}")
return None
# Example usage:
text = "Hello world"
response = embed_content(api_key, text)
if response:
print(response)
# test the code
for _ in tqdm(range(2000)):
embed_content(api_key, text)
```
https://ai.google.dev/api/embeddings#endpoint
Making sure to follow these steps will guarantee the quickest resolution possible.
Thanks!
Contributor guide
Assessment
This issue has not been assessed yet.