google-deepmind / google-deepmind/gemma

Refactor SamplerLoop class into standalone functions

Open
#549 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
5.7k
Forks
1k
Avg merge
10h 33m
Merged PRs (30d)
2

Description

Current Behavior
gemma/gm/text/_sampler_loop.py
currently implements sampling as a
SamplerLoop
class (line 102-266). This class only contains static, hashable attributes and serves primarily as a namespace for related functions, which is not idiomatic Python for this use case.

There is a TODO at line 98-100:

python
# TODO(epot): Refactor into simple function, rather than class.
# * autoregressive_sample()
# * autoregressive_stream_sample()
The current class-based approach:

Adds unnecessary boilerplate (dataclass definition, self parameters)
Makes the code harder to understand (class usually implies stateful behavior)
Reduces functional programming benefits in a JAX-heavy codebase
Expected Behavior
The sampling logic should be refactored into two standalone, pure functions that better align with JAX's functional programming paradigm:

autoregressive_sample() - Non-streaming sampling
autoregressive_stream_sample() - Streaming sampling
This would simplify the API, improve code clarity, and make it easier to compose with other JAX transformations.

Proposed Solution
I will refactor _sampler_loop.py by:
Extract configuration parameters into a dedicated dataclass (e.g., SamplerConfig)
Convert SamplerLoop.sample() into autoregressive_sample() standalone function
Convert SamplerLoop._sample_loop() into internal helper or merge with main function
Convert SamplerLoop._stream_sample_loop() into autoregressive_stream_sample() standalone function
Update all call sites in
_sampler.py
_chat_sampler.py
, and
_tool_sampler.py
to use the new function-based API
Update tests in
_sampler_test.py
and
sampler_e2e_test.py
Environment
Gemma version: main branch
Python: 3.x (JAX-compatible)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.