HarperFast / HarperFast/rocksdb-js
Transaction-log purge works exactly once per process: remove_all orphans the open txn.state handle and wedges every later purge
- Dominant language
- C++
- Stars
- 21
- Forks
- 2
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 36
Description
## What happens
`delete_transaction_logs_before` (Harper) / `purgeLogs()` (rocksdb-js) works **exactly once per process**, then silently no-ops forever while transaction-log disk grows unbounded. Cycle 1 purges correctly (`entries_deleted: 1300, log_files_deleted: 2`); every later cycle returns success with `0`/`0` against long-closed, fully-rotated `.txnlog` files well before the requested cutoff, while disk climbs 0 → 17.86 → 35.71 → 53.57 → 71.42 MiB over five cycles (428.6 MiB over twenty-five). Only a process restart clears the wedge. This is the strongest root-cause candidate for harper#846's exact signature (including the restart-clears-it behavior).
Counter signature: `oldestSequenceNumber` goes 1 → 0 on the first purge, then wedges at 3 permanently while `current` climbs 4→5→6→7 and `fileCount` 2→3→4→5.
A second consequence of the same mechanism: once the store directory is nuked, same-process audit reads of a fully-purged idle table come back **empty** — the audit blast radius is schema-scoped (same-schema quiet table lost 30/30 audit entries when a sibling table's log purged; different-schema tables untouched) but **never crosses into primary data** (500/500 rows survive REST, `search_by_value`, and full scan across two clean restarts).
## Mechanism (on `origin/main` @ a941a670)
- `src/binding/transaction_log/transaction_log_store.cpp:766` — `doPurge()` calls `std::filesystem::remove_all(this->path)` once a store's `sequenceFiles` empties (an idle/fully-aged store), deleting the directory **containing `txn.state`** while the still-open `flushedStateFile` handle keeps writing to the unlinked inode.
- `transaction_log_store.cpp:1116` — `databaseFlushed()`'s reopen guard is `if (!this->flushedStateFile.is_open())` — it checks **openness, not existence**, so the state file is never recreated on disk.
- `getLastFlushedPosition()` (`transaction_log_store.cpp:349`) opens `txn.state` fresh by path on every call, so post-nuke it returns the `{0,0}` sentinel forever, and `doPurge`'s eligibility check (`sequenceNumber > lastFlushedPosition.logSequenceNumber`) never advances — every later purge no-ops.
Harness explanations were ruled out in-spec: the cutoff advances every cycle after an explicit flush, `purgeRuns` increments, and a raw `primaryStore.purgeLogs()` bypass of Harper's job queue returns the same zero — so this is not harper#2338's JS-scheduling bug (that PR fixes the never-re-armed `scheduleAuditCleanup` timer, the other half of the once-per-process signature). Reproduced independently at a different cycle count by a second agent.
## Existing work checked
- harper#846 (OPEN) — the umbrella retention issue; this mechanism is quoted there as a 2026-08-25 comment but has no dedicated issue.
- harper#2140 — closed 2026-08-28 as dup framing of #846, citing harper#2338 (OPEN, unmerged), which fixes only the JS re-arm half.
- rocksdb-js#799 (OPEN) — "Keep transaction-log retention on a durable sequence floor" protects the segment anchoring `txn.state` from purge, which would prevent the directory-nuke as a side effect, but does not name or fix the `is_open()` guard itself.
- No rocksdb-js issue matches `is_open`, `flushedStateFile`, or the reopen guard.
## What this does not prove
The mechanism-to-symptom chain for harper#846 is high-confidence but was established by source read + counter signature, not a patched-binary A/B. Audit-loss bounds were measured on RocksDB only.
---
*From dispatch QA findings F-218 / QA-815 / qa-wave-2026072616 (merged — one root cause), verified against rocksdb-js origin/main a941a670, 2026-08-29.*
Contributor guide
Research direction
Start with src/binding/transaction_log/transaction_log_store.cpp at doPurge() around line 766, getLastFlushedPosition() around line 349, and databaseFlushed() around line 1116. Reproduce repeated purgeLogs() calls after explicit flushes and inspect txn.state across directory removal; done means later eligible purges continue removing rotated logs without losing the state needed for future eligibility.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, node.js
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100