google / google/gemma.cpp

Page-size check prevents mmap for blob-aligned checkpoints

Open Beginner friendly
#1,031 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
7k
Forks
660
Avg merge
20h 43m
Merged PRs (30d)
33

Description

## Problem

`ChooseMode` disables mmap when the checkpoint size is not divisible by the system page size, even if the file is padded to the blob format's 256-byte alignment.

For example, the 4B checkpoint is 5,401,208,832 bytes: divisible by 256, but with a 2,048-byte remainder on a 4,096-byte page system. It therefore falls back to copying the weights into allocated memory instead of using file-backed mappings.

## Proposed fix

Relax the file-length alignment requirement from the base page size to `kBlobAlign` (256 bytes).

The OS accepts partial-page mappings, but inference kernels can read multiple vectors ahead. Retain the blob-alignment guard and fall back to ordinary reads for lengths that do not meet it. Arbitrary-length mmap support would require separate handling of those reads.

Covered by #1032.

Contributor guide

Open the contributing guide

Research direction

Start at ChooseMode and inspect the checkpoint length check that compares against the system page size, then trace the mmap and ordinary-read fallback paths. Done means blob-aligned lengths such as the 4B checkpoint can use file-backed mappings, while lengths that are not divisible by kBlobAlign still use ordinary reads.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
machine-learning, performance
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
75/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.