googleapis / googleapis/python-genai

Vertex API does not return input_token_limit, Gemini API does

Offen
#574 0 Kommentare 1 Reaktion 1 zugewiesene Person Beansprucht von @sasha-gitg Auf GitHub ansehen
priority: p3 type: question
Vorherrschende Sprache
Python
Sterne
4k
Forks
1k
Ø Merge
2 T. 12 Std.
Gemergte PRs (30 T.)
41

Beschreibung

I have been using the Gemini API for some prototyping. I was delighted that the API exposes the input and output context window sizes. Using e.g. the OpenAI API I have to keep track of context window sizes myself. Since my prototyping seems promising (and I miss some of the JSON schema capabilities in the vertex API not supported in the Gemini API) I want to use the Vertex API instead of the Gemini API.

In Python using the genmien API this works beautifully:

```python
client = Client(api_key=self._configuration.gemini_api_key)
model_properties = client.models.get(model=model_name)
max_input_tokens = model_properties.input_token_limit
```

But when I use the Vertex API (setting required env vars and logging through the cli):

```python
client = Client()
model_properties = client.models.get(model=model_name)
```
`model_properties` does not contain `input_token_limit`. Clearly, this is intentional (`models.py:4830`):

```python
if self._api_client.vertexai:
response_dict = _Model_from_vertex(self._api_client, response_dict)
else:
response_dict = _Model_from_mldev(self._api_client, response_dict)
```

My question is whether you know if these model properties will become available in the Vertex API at some stage?

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.