abetlen / abetlen/llama-cpp-python

Request for Gemma 3 4B & Gemma 4 Integration Support with llama-cpp-python

Open
#2,231 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
10.6k
Forks
1.4k
PR merge metrics
PR metrics pending

Description

Hi team,

I would like to request official support and documentation for Gemma 3 4B and Gemma 4 integration within llama-cpp-python.

It would be helpful to have clarification regarding:

* Current compatibility status
* Required llama.cpp / llama-cpp-python versions
* Chat template handling requirements
* Recommended GGUF formats and quantizations
* Known limitations or issues
* Recommended inference parameters for stable usage

Any guidance, implementation notes, or examples would be greatly appreciated (without using ollama)

Kind regards,
Jonathan Park

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.