google / google/gemma.cpp

Add SmolLM2 support — reuse the Qwen3 converter path?

Open
#982 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
7k
Forks
660
Avg merge
20h 43m
Merged PRs (30d)
33

Description

Hi @jan-wassenberg — I'd like to contribute SmolLM2 (135M / 360M / 1.7B) support, and wanted to check the approach with you before writing any code.

SmolLM2 is plain Llama-style: RMSNorm, SwiGLU, RoPE theta 10000, no biases, no QK-norm, tied embeddings, byte-level BPE, ChatML turn tokens. As far as I can tell that needs **no new kernels** — everything is already on the Qwen3 path.

Sketch:

1. `python/convert_from_safetensors.py` — the HF tensor names are identical to Qwen3's, so `export_qwen3_lm_sbs` almost works as-is. The only blockers are the `has_qk_norm` assert and deriving `head_dim` from `q_norm.weight`. Plus a `smollm2-*` dispatch prefix.
2. `gemma/configs.{h,cc}` — new `Model` enum values + config functions.
3. `gemma/tokenizer.cc` — reuse the Qwen3 branch (same `<|im_start|>` / `<|im_end|>`).
4. `gemma/gemma.cc` — `HasEmbeddingScaling()` has to return false for it.

Questions:

- Is a third family outside Gemma/Qwen welcome here, or would you rather keep the model list narrow?
- Prefer a family-neutral `export_llama_style_lm_sbs` that both Qwen3 and SmolLM2 route through, or a separate function?
- For `HasEmbeddingScaling`, would you rather grow the per-family check, or add a `ModelConfig` field?

Happy to send a PR if the direction sounds right.

Contributor guide

Open the contributing guide

Research direction

Read python/convert_from_safetensors.py and the existing Qwen3 path first, then inspect gemma/configs.{h,cc}, gemma/tokenizer.cc, and gemma/gemma.cc for model registration, token handling, and embedding scaling. Done means the maintainers have chosen the family and converter design, and SmolLM2 135M, 360M, and 1.7B support is implemented without new kernels.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.