googleapis / googleapis/python-genai

Vertex API does not return input_token_limit, Gemini API does

Abierto
#574 0 comentarios 1 reacción 1 asignado Reclamado por @sasha-gitg Ver en GitHub
priority: p3 type: question
Lenguaje dominante
Python
Estrellas
4k
Forks
1k
Merge medio
2 d 12 h
PR fusionados (30 d)
41

Descripción

I have been using the Gemini API for some prototyping. I was delighted that the API exposes the input and output context window sizes. Using e.g. the OpenAI API I have to keep track of context window sizes myself. Since my prototyping seems promising (and I miss some of the JSON schema capabilities in the vertex API not supported in the Gemini API) I want to use the Vertex API instead of the Gemini API.

In Python using the genmien API this works beautifully:

```python
client = Client(api_key=self._configuration.gemini_api_key)
model_properties = client.models.get(model=model_name)
max_input_tokens = model_properties.input_token_limit
```

But when I use the Vertex API (setting required env vars and logging through the cli):

```python
client = Client()
model_properties = client.models.get(model=model_name)
```
`model_properties` does not contain `input_token_limit`. Clearly, this is intentional (`models.py:4830`):

```python
if self._api_client.vertexai:
response_dict = _Model_from_vertex(self._api_client, response_dict)
else:
response_dict = _Model_from_mldev(self._api_client, response_dict)
```

My question is whether you know if these model properties will become available in the Vertex API at some stage?

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.