Node cache is a fixed 16384 entries: no scan reuse above ~60 MB, and PRAGMA cache_size cannot lower it
- Dominant language
- C
- Stars
- 268
- Forks
- 18
- Avg merge
- 2h 26m
- Merged PRs (30d)
- 454
Description
Found in a full-repo review at `0ba280f06f`.
The prolly node cache is fixed at 16384 entries and `PRAGMA cache_size` can only raise it, never lower it. That produces two opposite problems: above roughly 60 MB of chunks a repeated scan gets **no reuse at all**, and below that the process still holds ~100 MB that the application cannot give back.
## Symptom 1: reuse collapses above the capacity
Three identical `SELECT count(s) FROM t` in one connection, 3M rows / 169 MB:
```
default node cache : 0.795 s 0.800 s 0.798 s <- no reuse whatsoever
PRAGMA cache_size=300000 : 0.791 s 0.085 s 0.085 s <- 9.3x on the warm runs
```
It is a cliff, not a gradient — reuse works fine until the table outgrows the cache and then stops entirely:
| rows | file | cold | warm | warm |
|---|---|---|---|---|
| 300,000 | 16 MB | 0.0277 | 0.0082 | 0.0080 |
| 700,000 | 39 MB | 0.0670 | 0.0197 | 0.0188 |
| 1,500,000 | 84 MB | 0.390 | **0.398** | **0.398** |
| 3,000,000 | 169 MB | 0.795 | **0.800** | **0.798** |
16384 entries × ~3.7 KB mean chunk ≈ 60 MB, which is where the cliff sits.
The warm 3M scan at `cache_size=300000` is 0.085 s against stock's 0.140 s for the same query — **the scan engine is faster than stock once the cache holds the working set.** The whole gap is cache sizing.
## Symptom 2: the floor cannot be lowered
Peak RSS, `SELECT count(s) FROM t` on 2M rows:
```
doltlite, default 99.9 MB
doltlite, PRAGMA cache_size=100 100.5 MB <- ignored
doltlite, PRAGMA cache_size=-2000 97.1 MB <- ignored
stock, default 5.6 MB
stock, PRAGMA cache_size=100 3.8 MB
```
Write transactions buffer everything as well — a 1M-row single-transaction `INSERT` peaks at **105.8 MB** versus stock's **3.4 MB**, because stock spills and DoltLite does not. That bounds the largest transaction that fits in memory, and it is a hard constraint on wasm and mobile.
## Cause
`prollyBtreeSetCacheSize` (`src/prolly_btree.c`) clamps every request up to the baseline:
```c
/* Never shrink below the engine baseline (stock default is -2000). */
if( nEntry < PROLLY_DEFAULT_CACHE_SIZE ) nEntry = PROLLY_DEFAULT_CACHE_SIZE;
p->pBt->cache.nCapacity = (int)nEntry;
```
with `PROLLY_DEFAULT_CACHE_SIZE 16384` (`src/prolly_btree_int.h:130`). So the capacity is a fixed entry count, not a byte budget: it is far too small for a 169 MB database and far too large for an embedded process that wanted 2 MB.
A likely complementary cause, identified during the review but not separately isolated by me: the on-disk chunk index has no page cache of its own — `csIndexLookup` (`src/chunk_index.c:206-300`) walks the checkpointed index from the root on every chunk fetch, doing a `pread`, a `sqlite3_malloc` and a full BLAKE3 over each 4 KB page on the path, with nothing retained between lookups (`csReadLazyPage`, `src/chunk_index.c:122-163`). That would explain why a cache miss is expensive rather than merely a re-read.
## Fix
Make the capacity a **byte budget that honours `cache_size` in both directions**, with a small absolute minimum rather than a 16384-entry floor — that fixes both symptoms with one change. Separately, consider caching validated chunk-index pages keyed by file offset (hash-verified once at fill, invalidated on checkpoint) so that a miss costs a read rather than a re-hash of the whole root-to-leaf path. Spilling mutmap edits past a budget would address the write-side number.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with prollyBtreeSetCacheSize in src/prolly_btree.c and PROLLY_DEFAULT_CACHE_SIZE in src/prolly_btree_int.h, then inspect the reported scan and PRAGMA cache_size behavior. Compare cache capacity in both directions against the benchmark results; done means cache sizing is a byte budget that can be raised or lowered, with the chunk-index caching and write-spilling ideas treated as separate work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, sqlite
- Domain
- databases, performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100