googleapis / googleapis/python-genai

`local-tokenizer` extra installs the full PyTorch/CUDA stack for SentencePiece-only models

Open
#2,728 4 comments 2 reactions 1 assignee Claimed by @Venkaiahbabuneelam View on GitHub
priority: p2 type: bug
Dominant language
Python
Stars
4k
Forks
1k
Avg merge
2d 11h
Merged PRs (30d)
40

Description

### Description

Installing `google-genai[local-tokenizer]` installs Pillow, PyTorch, Torchvision, Transformers, Triton and the Linux NVIDIA CUDA packages for every locally tokenised model.

However, models such as `gemini-2.5-flash` use the lightweight SentencePiece path. The Hugging Face `AutoProcessor` path is selected only for the Gemma 4 model mappings.

This makes local tokenisation unexpectedly expensive for applications that do not use Gemma 4.

### Reproduction

Environment:

- Linux x86_64
- Python 3.14.2
- uv
- `google-genai==2.11.0`

```bash
uv venv --python 3.14 base
uv pip install --python base/bin/python \
'google-genai[aiohttp]==2.11.0'

uv venv --python 3.14 local
uv pip install --python local/bin/python \
'google-genai[aiohttp,local-tokenizer]==2.11.0'

du -sh base local
```

Result:

```text
45M base
4.7G local
```

Largest installed components:

```text
1.1G torch
2.7G nvidia
689M triton
13M torchvision
```

The dependency count increases from 33 packages to 83.

### Expected behaviour

Applications using the SentencePiece-backed local tokenizer should not need to install the Gemma 4 Hugging Face and PyTorch stack.

Could the extras be split, for example:

```toml
local-tokenizer = [
"sentencepiece>=0.2.0",
"protobuf",
]

local-tokenizer-gemma4 = [
"sentencepiece>=0.2.0",
"protobuf",
"pillow",
"torch",
"torchvision",
"transformers",
]
```

The implementation already imports `AutoProcessor` lazily inside the Gemma 4 path, so the runtime boundary largely exists.

It may also be worth checking whether `AutoTokenizer` could replace `AutoProcessor` for this text-only use case, potentially removing some image-related dependencies. That requires validation against Gemma 4.

### Downstream impact

This caused one downstream production service environment to increase from approximately 626 MiB to 5.1 GiB. We had to remove the local-tokenizer extra from our shared library's base installation and fall back to the remote `count_tokens` endpoint.

Relevant code:

- [v2.11.0 optional dependencies](https://github.com/googleapis/python-genai/blob/v2.11.0/pyproject.toml)
- [Local tokenizer loader](https://github.com/googleapis/python-genai/blob/v2.11.0/google/genai/_local_tokenizer_loader.py)
- [Gemma 4 implementation PR](https://github.com/googleapis/python-genai/pull/2576)
- [Dependency expansion PR](https://github.com/googleapis/python-genai/pull/2590)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.