googleapis / googleapis/python-genai

`local-tokenizer` extra installs the full PyTorch/CUDA stack for SentencePiece-only models

Aperta
#2,728 4 commenti 2 reazioni 1 assegnatario Rivendicata da @Venkaiahbabuneelam Vedi su GitHub
priority: p2 type: bug
Lingua principale
Python
Stelle
4k
Fork
1k
Merge medio
2g 12h
PR unite (30g)
41

Descrizione

### Description

Installing `google-genai[local-tokenizer]` installs Pillow, PyTorch, Torchvision, Transformers, Triton and the Linux NVIDIA CUDA packages for every locally tokenised model.

However, models such as `gemini-2.5-flash` use the lightweight SentencePiece path. The Hugging Face `AutoProcessor` path is selected only for the Gemma 4 model mappings.

This makes local tokenisation unexpectedly expensive for applications that do not use Gemma 4.

### Reproduction

Environment:

- Linux x86_64
- Python 3.14.2
- uv
- `google-genai==2.11.0`

```bash
uv venv --python 3.14 base
uv pip install --python base/bin/python \
'google-genai[aiohttp]==2.11.0'

uv venv --python 3.14 local
uv pip install --python local/bin/python \
'google-genai[aiohttp,local-tokenizer]==2.11.0'

du -sh base local
```

Result:

```text
45M base
4.7G local
```

Largest installed components:

```text
1.1G torch
2.7G nvidia
689M triton
13M torchvision
```

The dependency count increases from 33 packages to 83.

### Expected behaviour

Applications using the SentencePiece-backed local tokenizer should not need to install the Gemma 4 Hugging Face and PyTorch stack.

Could the extras be split, for example:

```toml
local-tokenizer = [
"sentencepiece>=0.2.0",
"protobuf",
]

local-tokenizer-gemma4 = [
"sentencepiece>=0.2.0",
"protobuf",
"pillow",
"torch",
"torchvision",
"transformers",
]
```

The implementation already imports `AutoProcessor` lazily inside the Gemma 4 path, so the runtime boundary largely exists.

It may also be worth checking whether `AutoTokenizer` could replace `AutoProcessor` for this text-only use case, potentially removing some image-related dependencies. That requires validation against Gemma 4.

### Downstream impact

This caused one downstream production service environment to increase from approximately 626 MiB to 5.1 GiB. We had to remove the local-tokenizer extra from our shared library's base installation and fall back to the remote `count_tokens` endpoint.

Relevant code:

- [v2.11.0 optional dependencies](https://github.com/googleapis/python-genai/blob/v2.11.0/pyproject.toml)
- [Local tokenizer loader](https://github.com/googleapis/python-genai/blob/v2.11.0/google/genai/_local_tokenizer_loader.py)
- [Gemma 4 implementation PR](https://github.com/googleapis/python-genai/pull/2576)
- [Dependency expansion PR](https://github.com/googleapis/python-genai/pull/2590)

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.