leanprover / leanprover/lean4

olean loader: no-mmap fallback re-reads the entire file after the header probe

Open
#14,991 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug P-low
Dominant language
Lean
Stars
9.2k
Forks
990
Avg merge
1d 17h
Merged PRs (30d)
175

Description

In lean_compacted_region_read

  1. The 88-byte olean_header is read to validate magic/version/githash
    (readn(fd.get(), &header, sizeof(header)), ~line 482).
  2. If LEAN_MMAP is compiled in, the file is mmapped at the saved base_addr.
  3. On the fallback — no LEAN_MMAP, or the mapping did not land at
    base_addr — the loader does malloc(size), lseek(fd, 0, SEEK_SET),
    and re-reads the whole file from offset 0 (~lines 558–560), including
    the header bytes it already consumed.

I am working on Lean in WASM64 which is configured with -DMMAP=OFF takes the
fallback for every olean. On Emscripten targets the redundant second read goes
through proxied filesystem emulation; an import trace there measured
38,568 read operations for 19,284 module loads — exactly 2× per module.
Desktop builds also hit the fallback whenever the mmap address check
fails at runtime.

strace -f -e trace=read,lseek lean SomeFile.lean 2>&1 | grep -B1 -A2 '", 88) = 88'

On fallback loads the same fd shows the 88-byte header read followed by
lseek(fd, 0, SEEK_SET) and a full-file read.
I do have a AI assisted patch here if needed https://github.com/FawadHa1der/lean4/commit/2dab7cd11c (guarded by
#ifndef LEAN_MMAP); happy to adapt it into a PR.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/library/module.cpp at lean_compacted_region_read and inspect the 88-byte olean_header probe and the fallback around lines 558–560. Use the reported strace command, or an equivalent WASM64 import trace, to verify that fallback loads no longer reread the header while still loading the full file.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers, performance
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.