google / google/gemma.cpp

QAT Models supported?

Open
#543 5 comments 2 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
7k
Forks
660
Avg merge
20h 43m
Merged PRs (30d)
33

Description

Thanks for the wonderful repo. It's a pleasure to work with it.

Are the new 4-bit quantized models for gemma 3 supported by gemma.cpp? Judging from the press-release https://developers.googleblog.com/en/gemma-3-quantized-aware-trained-state-of-the-art-ai-to-consumer-gpus/ and looking through the gemma.cpp repo, it doesn't seem like it, but I am bit surprised about it.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.