googleapis / googleapis/python-genai

Vertex API does not return input_token_limit, Gemini API does

Ouverte
#574 0 commentaires 1 réaction 1 personne assignée Réclamée par @sasha-gitg Voir sur GitHub
priority: p3 type: question
Langage dominant
Python
Étoiles
4k
Forks
1k
Merge moyen
2 j 12 h
PR mergées (30 j)
41

Description

I have been using the Gemini API for some prototyping. I was delighted that the API exposes the input and output context window sizes. Using e.g. the OpenAI API I have to keep track of context window sizes myself. Since my prototyping seems promising (and I miss some of the JSON schema capabilities in the vertex API not supported in the Gemini API) I want to use the Vertex API instead of the Gemini API.

In Python using the genmien API this works beautifully:

```python
client = Client(api_key=self._configuration.gemini_api_key)
model_properties = client.models.get(model=model_name)
max_input_tokens = model_properties.input_token_limit
```

But when I use the Vertex API (setting required env vars and logging through the cli):

```python
client = Client()
model_properties = client.models.get(model=model_name)
```
`model_properties` does not contain `input_token_limit`. Clearly, this is intentional (`models.py:4830`):

```python
if self._api_client.vertexai:
response_dict = _Model_from_vertex(self._api_client, response_dict)
else:
response_dict = _Model_from_mldev(self._api_client, response_dict)
```

My question is whether you know if these model properties will become available in the Vertex API at some stage?

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.