googleapis / googleapis/python-genai
`local-tokenizer` extra installs the full PyTorch/CUDA stack for SentencePiece-only models
- Dominant language
- Python
- Stars
- 4k
- Forks
- 1k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 40
Description
### Description
Installing `google-genai[local-tokenizer]` installs Pillow, PyTorch, Torchvision, Transformers, Triton and the Linux NVIDIA CUDA packages for every locally tokenised model.
However, models such as `gemini-2.5-flash` use the lightweight SentencePiece path. The Hugging Face `AutoProcessor` path is selected only for the Gemma 4 model mappings.
This makes local tokenisation unexpectedly expensive for applications that do not use Gemma 4.
### Reproduction
Environment:
- Linux x86_64
- Python 3.14.2
- uv
- `google-genai==2.11.0`
```bash
uv venv --python 3.14 base
uv pip install --python base/bin/python \
'google-genai[aiohttp]==2.11.0'
uv venv --python 3.14 local
uv pip install --python local/bin/python \
'google-genai[aiohttp,local-tokenizer]==2.11.0'
du -sh base local
```
Result:
```text
45M base
4.7G local
```
Largest installed components:
```text
1.1G torch
2.7G nvidia
689M triton
13M torchvision
```
The dependency count increases from 33 packages to 83.
### Expected behaviour
Applications using the SentencePiece-backed local tokenizer should not need to install the Gemma 4 Hugging Face and PyTorch stack.
Could the extras be split, for example:
```toml
local-tokenizer = [
"sentencepiece>=0.2.0",
"protobuf",
]
local-tokenizer-gemma4 = [
"sentencepiece>=0.2.0",
"protobuf",
"pillow",
"torch",
"torchvision",
"transformers",
]
```
The implementation already imports `AutoProcessor` lazily inside the Gemma 4 path, so the runtime boundary largely exists.
It may also be worth checking whether `AutoTokenizer` could replace `AutoProcessor` for this text-only use case, potentially removing some image-related dependencies. That requires validation against Gemma 4.
### Downstream impact
This caused one downstream production service environment to increase from approximately 626 MiB to 5.1 GiB. We had to remove the local-tokenizer extra from our shared library's base installation and fall back to the remote `count_tokens` endpoint.
Relevant code:
- [v2.11.0 optional dependencies](https://github.com/googleapis/python-genai/blob/v2.11.0/pyproject.toml)
- [Local tokenizer loader](https://github.com/googleapis/python-genai/blob/v2.11.0/google/genai/_local_tokenizer_loader.py)
- [Gemma 4 implementation PR](https://github.com/googleapis/python-genai/pull/2576)
- [Dependency expansion PR](https://github.com/googleapis/python-genai/pull/2590)
Contributor guide
Assessment
This issue has not been assessed yet.