HarperFast / HarperFast/harper

LMDB: TTL eviction of a record with a Blob attribute orphans the blob file on disk (unbounded disk growth; RocksDB reclaims correctly)

Open
#1,606 0 comments 0 reactions 0 assignees View on GitHub
area:storage bug
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 6h
Merged PRs (30d)
200

Description

## Summary

On the **LMDB** storage engine, when a record with a `Blob` attribute expires via TTL, LMDB removes the row but leaves the blob **file** on disk. The orphaned files are never reclaimed → **unbounded disk growth** for any LMDB table combining TTL (`expiration`) with blob attributes. **RocksDB reclaims correctly** — this is engine-divergent.

## Reproduction (harper main `228eacc0f`, v5.1.15)

- Table `@table(expiration: 5)` with a `Blob` attribute, `threads: { count: 4 }`.
- Insert 5 records, each with a distinct ~200 KB blob.
- Confirm eviction: point-read → 404 (~4 s after the deadline) **before** counting.
- Count blob files under `{dataRootDir}/blobs/data` before vs after eviction.

| Engine | before → after |
|---|---|
| RocksDB | 5 → 0 (clean); replace-then-expire 1 → 0 (both v1 + v2 reclaimed) |
| **LMDB** | 5 → **5** (leak) |

Re-ran the LMDB case with a 30 s settle → 5 → **1**, not 0 — so this is **not** eviction lag; LMDB's unlink is partial/racy and permanently orphans blob files.

## Root-cause hypothesis (unverified, medium confidence)

`Table.evict() → removeEntry()` gates blob deletion on `entry.value && (metadataFlags & HAS_BLOBS)`, but the LMDB eviction path's `getEntry(id)` doesn't reliably return a populated value/flags, so `deleteBlobsInObject` never fires. Candidate fix: feed the scan's already-decoded record into `evict()` rather than re-fetching it.

## Severity

MEDIUM. LMDB is the deprecated engine (replication is RocksDB-only), so blast radius is limited to LMDB deployments — but it is a silent, unbounded-disk-growth footgun for anyone on LMDB with blob + TTL tables (sessions, caches, media with expiry).

Found via exploratory QA; local repro test available (`qa-scratch/qa-blob-ttl.test.ts`).

— KrAIs 🤖 (exploratory QA, on Kris's behalf)

Contributor guide

Open the contributing guide

Research direction

Start with the local reproduction in qa-scratch/qa-blob-ttl.test.ts, then trace Table.evict() through removeEntry(), getEntry(id), and deleteBlobsInObject. Verify the LMDB TTL path reclaims blob files after eviction, including the reported replacement case, rather than leaving files under {dataRootDir}/blobs/data.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, nodejs
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.