google-deepmind / google-deepmind/gemma

Support Gemma 4 12B Unified in the JAX package

Open
#709 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
5.7k
Forks
1k
Avg merge
10h 33m
Merged PRs (30d)
2

Description

Hi Gemma team,

Thanks for maintaining this package. I am opening this as a support / enhancement question rather than a bug report; apologies if I missed an existing supported path or a duplicate issue.

I am trying to use the released Gemma 4 12B Unified model from the JAX `gemma` package. In my checkout of current `main` (`05652d59d546d3822b3dfc1f60133cabfe265cbb`), Gemma 4 support appears to expose these model classes:

- `gm.nn.Gemma4_E2B`
- `gm.nn.Gemma4_E4B`
- `gm.nn.Gemma4_26B_A4B`
- `gm.nn.Gemma4_31B`

and matching checkpoint enum entries:

- `CheckpointPath.GEMMA4_E2B_*`
- `CheckpointPath.GEMMA4_E4B_*`
- `CheckpointPath.GEMMA4_26B_A4B_*`
- `CheckpointPath.GEMMA4_31B_*`

I did not find a `Gemma4_12B`, `GEMMA4_12B_*`, `gemma4-12b`, or `12B Unified` equivalent in the JAX package surface.

The public model appears to be available as:

- https://huggingface.co/google/gemma-4-12B
- https://huggingface.co/google/gemma-4-12B-it
- https://ai.google.dev/gemma/docs/core/model_card_4

Would you be open to adding first-class JAX support for this variant, or documenting the intended load path if it already exists?

Specifically, it would be helpful to know:

1. Is Gemma 4 12B Unified support planned for this repo?
2. Should users expect a new `gm.nn.Gemma4_12B` class and `CheckpointPath.GEMMA4_12B_*` enum entries, or a different naming/API?
3. Is there an interim way to load the 12B Unified Hugging Face checkpoint with this package?

Why I am asking: Gemma 4 12B Unified seems to have different architecture/runtime assumptions from the currently exposed Gemma 4 variants, and downstream projects are already tracking support issues for it. A canonical JAX path or doc note here would help users avoid guessing.

Thanks!

Contributor guide

Open the contributing guide

Research direction

Start by inspecting the existing gm.nn Gemma 4 classes and CheckpointPath.GEMMA4_* entries mentioned in the issue, then compare them with the released Gemma 4 12B checkpoints. Done means either first-class loading support is implemented and tested for the 12B Unified variants, or the intended Hugging Face load path is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, python
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.