google / google/gemma.cpp

[User Experience Failure] Impossible to Build and Run on a Resource-Constrained Google Edge Device (Chromebook)

Open
#688 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
7k
Forks
660
Avg merge
20h 43m
Merged PRs (30d)
33

Description

I am a (hobby) programmer with 40 years of experience...blah. it's actually 45 years. My first machine was the TRS80. By 1999 I was compiling Allegro Games on 500 MHz. Logically, I am that dinosaur that would gravitate towards gemma.cpp

I tend to isolate and don't socialize much online. New to GitHub.

My goal was to follow the official documentation to compile and run a Gemma 3 model on a standard, Google-made edge device: an Acer 715 Chromebook with the Linux Development Environment. This is the exact use case for which gemma.cpp is marketed.

The official workflow for an on-device build is fundamentally broken for a resource-constrained environment. I have spent multiple days navigating a series of undocumented dependencies, build failures, and broken documentation. The process failed at every critical stage.

Build Process Failure: The standard build process fails with an Out-of-Memory (OOM) error on a standard Chromebook. The compiler is killed by the OS.

Build Process Workaround Failure: Forcing a single-threaded build (-j 1) also fails with an OOM error, proving that even single source files have memory requirements that exceed the capacity of a standard edge device.

Partial Build Catastrophe: The only way to compile the C++ executable is to disable tests (-DGEMMA_ENABLE_TESTS=OFF). However, this silently and catastrophically fails to build the required Python libraries (compression.python), making the necessary convert_from_safetensors.py script non-functional with an ImportError. This is a critical, undocumented failure of the build system.

Documentation Failure (Missing Files): The official documentation directs users to download pre-converted .sbs/.sfp model files from Kaggle. These files do not exist for the smaller 1B/4B models. The documentation is a lie.

Dependency Hell: The Python conversion script has numerous undocumented dependencies (numpy, torch, sentencepiece, etc.). The requirements.txt file is woefully incomplete, forcing users into a painful, multi-step ModuleNotFoundError loop.

Bloatware Requirement: The conversion process requires installing the entire, multi-gigabyte PyTorch library (including useless NVIDIA drivers) on a small edge device for what should be a simple file format conversion. This is a complete betrayal of the project's goal of being lightweight and efficient.

As it stands, gemma.cpp is not a usable tool for its target audience on its target hardware. It is impossible to correctly compile the full project and its dependencies on a standard Chromebook. The documentation is incorrect, and the workflow is hostile to the user.

For this Google project to fulfill its mission, it must provide one of two things:

A build process that can actually be completed on a resource-constrained device.

The pre-converted .sbs/.sfp model files for all model sizes, as the documentation incorrectly promises.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the Chromebook workflow described in the issue, including the build with -j 1 and -DGEMMA_ENABLE_TESTS=OFF. Inspect requirements.txt and convert_from_safetensors.py, then verify the documented Kaggle model files and Python dependencies. Done means the documented build and conversion workflow works on the target device, or its limitations and required artifacts are documented accurately.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python, pytorch
Domain
build-system, documentation, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.