googleapis / googleapis/python-genai

google.genai only support batch embedding which leads to rate limit errors

Aperta
#427 2 commenti 4 reazioni 1 assegnatario Rivendicata da @jaycee-li Vedi su GitHub
priority: p2 type: feature request
Lingua principale
Python
Stelle
4k
Fork
1k
Merge medio
2g 12h
PR unite (30g)
41

Descrizione

Thanks for stopping by to let us know something could be better!

**PLEASE READ**: If you have a support contract with Google, please create an issue in the [support console](https://cloud.google.com/support/) instead of filing on GitHub. This will ensure a timely response.

Is this a client library issue or a product issue? We will only be able to assist with issues that pertain to the behaviors of this library. If the issue you're experiencing is due to the behavior of the product itself, please visit the [Support page](https://cloud.google.com/support) to reach the most relevant engineers.

If the support paths suggested above still do not result in a resolution, please provide the following details.

#### Environment details

- Programming language: Python
- OS: MacOS
- Language runtime version: 3.11
- Package version: google-genai==1.3.0

#### Steps to reproduce

1. Using a paid API
2. Run an embeddings job (sync)

```python

from tqdm import tqdm
import time

client = genai.Client(api_key=api_key)
error_counter = 0

for _ in tqdm(range(2000)):
success = False
while not success:
try:
response = client.models.embed_content(
model='text-embedding-004',
contents='Hello world',
)
success = True
except Exception as e:
print(e)
error_counter += 1
if "429" in (str(e)):
print("sleeping....")
time.sleep(10)

```

The function `embed_content` uses the following logic

```
path = '{model}:batchEmbedContents'.format_map(request_url_dict)
```

https://github.com/googleapis/python-genai/blob/91b1d3ee85fd32e9243e3a2da4d10a763e4d0005/google/genai/models.py#L4399

## The FIX

Please implement the following for users so that they stop hitting that 150 batch limit rate

```python

import requests
import json

def embed_content(api_key, text, model="text-embedding-004"):
"""Embeds text using the Gemini API.

Args:
api_key: Your Google Cloud API key.
text: The text to embed.
model: The embedding model to use.

Returns:
The JSON response from the API, or None if there was an error.
"""
url = f"https://generativelanguage.googleapis.com/v1beta/models/{model}:embedContent?key={api_key}"
headers = {"Content-Type": "application/json"}
data = {
"model": model,
"content": {
"parts": [{"text": text}]
}
}

try:
response = requests.post(url, headers=headers, data=json.dumps(data))
response.raise_for_status() # Raise an exception for error responses
return response.json()
except requests.exceptions.RequestException as e:
print(f"Error embedding content: {e}")
return None

# Example usage:

text = "Hello world"
response = embed_content(api_key, text)

if response:
print(response)

# test the code

for _ in tqdm(range(2000)):
embed_content(api_key, text)

```

https://ai.google.dev/api/embeddings#endpoint

Making sure to follow these steps will guarantee the quickest resolution possible.

Thanks!

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.