HarperFast / HarperFast/harper
restore_backup rolls a database back in time with no epoch, so audit cursors, record versions, and replication sequence state all read as valid afterwards
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 200
Description
`restore_backup` rolls a database back in time, and nothing tells a consumer holding state from after the backup point that its world changed. Several independent pieces of resumable state are affected, so each one currently looks valid on its own:
- **Audit retention floor.** `restore_backup` replaces the database directory wholesale (`dataLayer/rocksdbBackup.ts:574` → `backups.restore(..., { mode: 'purgeAllFiles' })`), so the restored database carries the *backup's* floor. A consumer that had advanced past the backup point compares its cursor against that older floor, reads it as safe, resumes — and waits for entries that no longer exist. Found reviewing #2447, which added the floor.
- **Record versions.** Restored records carry the backup's versions, so a peer or cache that saw newer versions now holds state the restored database will not converge to on its own.
- **Per-node replication sequence state.** `Symbol.for('seq')` records (`resources/nodeIdMapping.ts`) come back at their backup values, so sequence positions a peer already acknowledged are reissued.
The audit floor could be stamped with a fresh epoch during restore in isolation, and #2447 deliberately did not: making one of these three honest while the other two stay stale gives a consumer a *more* confusing answer, not a safer one — it would be told to resync the audit stream while replication and cached versions silently disagree.
## What this needs
A **database-level generation/epoch** that increments on restore, and that resumable state can be compared against:
- established at a single point in the restore sequence, after engine restoration and before `completeRestore` / worker republication (`dataLayer/rocksdbBackup.ts:574-605`), so no thread can observe the restored database at the old generation;
- readable by the things that hold cursors, so "your state predates the current generation" is answerable without each subsystem inventing its own marker;
- applied to at least the three cases above, with the audit floor's treatment being "the generation changed, so the floor is unknown" rather than a fabricated timestamp.
Worth deciding as part of the design: whether an offline (CLI) restore and an online `restore_backup` must behave identically here, and whether a restore should be *visible* to clients (an error on resume) or merely force a resync.
## Acceptance
- After a restore, a consumer resuming from a cursor recorded before the restore but after the backup point is told its state is stale rather than receiving a silently empty or short stream.
- An integration test covering backup → advance a consumer's cursor → restore → reopen, asserting the resume is refused.
- The three state carriers above either consult the generation or are explicitly documented as out of scope with the reason.
## Related
- #2447 — added the audit retention floor; documents this gap in `getAuditFloor`'s contract rather than papering over it.
- #2448 — consumes the floor inside `Table.subscribe`; whatever this issue lands will need to be visible on the same resume path.
Contributor guide
Research direction
Start at dataLayer/rocksdbBackup.ts:574-605 and trace restore through completeRestore and worker republication, then inspect resources/nodeIdMapping.ts and the audit resume path used by Table.subscribe. Decide how a shared generation is exposed to resumable state, including offline CLI restore and client-visible stale errors. Done means an integration test covers backup, cursor advance, restore, and reopen, refusing the stale resume, with the three carriers addressed or explicitly scoped out.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, nodejs
- Domain
- backend, databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100