huggingface / huggingface/candle

Gemma 4 models cannot use

Open
#3,967 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
21k
Forks
1.8k
Avg merge
16h 42m
Merged PRs (30d)
25

Description

version: `ddf1b879dc3a1760cbcb3f3c4a7c6467850cec4a`
env: Ubuntu 24.04; CUDA 12.8

**Problem 1: Gemma 4 text checkpoint does not load**
The gemma4 example looks up `model.embed_tokens.weight`, but official Gemma 4 checkpoints store the language-model weights like `model.language_model.*.` On google/gemma-4-E4B-it that fails with cannot find tensor `model.embed_tokens.weight`

Image

**Problem 2: Gemma 4 E2B/E4B model does not work as it output random words**

Image

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by running the gemma4 example with the google/gemma-4-E4B-it checkpoint on the reported Ubuntu and CUDA environment, then inspect the model tensor lookup and generation behavior. Done means the official Gemma 4 E2B/E4B checkpoints load successfully and produce meaningful output rather than failing on model.embed_tokens.weight or generating random words.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.