googleapis / googleapis/python-genai

Vertex API does not return input_token_limit, Gemini API does

Aperta
#574 0 commenti 1 reazione 1 assegnatario Rivendicata da @sasha-gitg Vedi su GitHub
priority: p3 type: question
Lingua principale
Python
Stelle
4k
Fork
1k
Merge medio
2g 11h
PR unite (30g)
40

Descrizione

I have been using the Gemini API for some prototyping. I was delighted that the API exposes the input and output context window sizes. Using e.g. the OpenAI API I have to keep track of context window sizes myself. Since my prototyping seems promising (and I miss some of the JSON schema capabilities in the vertex API not supported in the Gemini API) I want to use the Vertex API instead of the Gemini API.

In Python using the genmien API this works beautifully:

```python
client = Client(api_key=self._configuration.gemini_api_key)
model_properties = client.models.get(model=model_name)
max_input_tokens = model_properties.input_token_limit
```

But when I use the Vertex API (setting required env vars and logging through the cli):

```python
client = Client()
model_properties = client.models.get(model=model_name)
```
`model_properties` does not contain `input_token_limit`. Clearly, this is intentional (`models.py:4830`):

```python
if self._api_client.vertexai:
response_dict = _Model_from_vertex(self._api_client, response_dict)
else:
response_dict = _Model_from_mldev(self._api_client, response_dict)
```

My question is whether you know if these model properties will become available in the Vertex API at some stage?

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.