googleapis / googleapis/python-genai

google.genai only support batch embedding which leads to rate limit errors

Offen
#427 2 Kommentare 4 Reaktionen 1 zugewiesene Person Beansprucht von @jaycee-li Auf GitHub ansehen
priority: p2 type: feature request
Vorherrschende Sprache
Python
Sterne
4k
Forks
1k
Ø Merge
2 T. 12 Std.
Gemergte PRs (30 T.)
41

Beschreibung

Thanks for stopping by to let us know something could be better!

**PLEASE READ**: If you have a support contract with Google, please create an issue in the [support console](https://cloud.google.com/support/) instead of filing on GitHub. This will ensure a timely response.

Is this a client library issue or a product issue? We will only be able to assist with issues that pertain to the behaviors of this library. If the issue you're experiencing is due to the behavior of the product itself, please visit the [Support page](https://cloud.google.com/support) to reach the most relevant engineers.

If the support paths suggested above still do not result in a resolution, please provide the following details.

#### Environment details

- Programming language: Python
- OS: MacOS
- Language runtime version: 3.11
- Package version: google-genai==1.3.0

#### Steps to reproduce

1. Using a paid API
2. Run an embeddings job (sync)

```python

from tqdm import tqdm
import time

client = genai.Client(api_key=api_key)
error_counter = 0

for _ in tqdm(range(2000)):
success = False
while not success:
try:
response = client.models.embed_content(
model='text-embedding-004',
contents='Hello world',
)
success = True
except Exception as e:
print(e)
error_counter += 1
if "429" in (str(e)):
print("sleeping....")
time.sleep(10)

```

The function `embed_content` uses the following logic

```
path = '{model}:batchEmbedContents'.format_map(request_url_dict)
```

https://github.com/googleapis/python-genai/blob/91b1d3ee85fd32e9243e3a2da4d10a763e4d0005/google/genai/models.py#L4399

## The FIX

Please implement the following for users so that they stop hitting that 150 batch limit rate

```python

import requests
import json

def embed_content(api_key, text, model="text-embedding-004"):
"""Embeds text using the Gemini API.

Args:
api_key: Your Google Cloud API key.
text: The text to embed.
model: The embedding model to use.

Returns:
The JSON response from the API, or None if there was an error.
"""
url = f"https://generativelanguage.googleapis.com/v1beta/models/{model}:embedContent?key={api_key}"
headers = {"Content-Type": "application/json"}
data = {
"model": model,
"content": {
"parts": [{"text": text}]
}
}

try:
response = requests.post(url, headers=headers, data=json.dumps(data))
response.raise_for_status() # Raise an exception for error responses
return response.json()
except requests.exceptions.RequestException as e:
print(f"Error embedding content: {e}")
return None

# Example usage:

text = "Hello world"
response = embed_content(api_key, text)

if response:
print(response)

# test the code

for _ in tqdm(range(2000)):
embed_content(api_key, text)

```

https://ai.google.dev/api/embeddings#endpoint

Making sure to follow these steps will guarantee the quickest resolution possible.

Thanks!

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.