olean loader: no-mmap fallback re-reads the entire file after the header probe
Nobody has claimed this yet.
- Dominant language
- Lean
- Stars
- 9.2k
- Forks
- 990
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 175
Description
- The 88-byte
olean_headeris read to validate magic/version/githash
(readn(fd.get(), &header, sizeof(header)), ~line 482). - If
LEAN_MMAPis compiled in, the file is mmapped at the savedbase_addr. - On the fallback — no
LEAN_MMAP, or the mapping did not land at
base_addr— the loader doesmalloc(size),lseek(fd, 0, SEEK_SET),
and re-reads the whole file from offset 0 (~lines 558–560), including
the header bytes it already consumed.
I am working on Lean in WASM64 which is configured with -DMMAP=OFF takes the
fallback for every olean. On Emscripten targets the redundant second read goes
through proxied filesystem emulation; an import trace there measured
38,568 read operations for 19,284 module loads — exactly 2× per module.
Desktop builds also hit the fallback whenever the mmap address check
fails at runtime.
strace -f -e trace=read,lseek lean SomeFile.lean 2>&1 | grep -B1 -A2 '", 88) = 88'
On fallback loads the same fd shows the 88-byte header read followed by
lseek(fd, 0, SEEK_SET) and a full-file read.
I do have a AI assisted patch here if needed https://github.com/FawadHa1der/lean4/commit/2dab7cd11c (guarded by
#ifndef LEAN_MMAP); happy to adapt it into a PR.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in src/library/module.cpp at lean_compacted_region_read and inspect the 88-byte olean_header probe and the fallback around lines 558–560. Use the reported strace command, or an equivalent WASM64 import trace, to verify that fallback loads no longer reread the header while still loading the full file.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- compilers, performance
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100