HarperFast / HarperFast/harper
Corrupt-frame transaction-log truncation is silent — no signal that a log tail was dropped
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 200
Description
## Summary
The corrupt-frame guard ([v5.2.1 `resources/replayLogsGuards.ts#L142`](https://github.com/HarperFast/harper/blob/v5.2.1/resources/replayLogsGuards.ts#L142), `endIteratorOnCorruptFrame`) latches iteration to `done` on a `RangeError` and logs once. That is the right availability call — replay and replication survive corruption — but it silently treats the corruption point as **end of that log file**: every entry after it is never replayed locally and never served to replication subscribers.
## Impact
After torn writes from an unclean shutdown (the common case for corruption), a node can boot green while permanently missing the tail of a log — and its subscribers miss it too. Nothing in `cluster_status`, health checks, or metrics distinguishes "recovered cleanly" from "recovered, dropped an unknown number of trailing entries; this node needs a re-clone." Operators have no signal that data recovery (vs availability recovery) is required.
## Suggested fix
Minimum: a loud, structured signal when the latch fires — counter in `system_information`/`cluster_status` (like `blobReplicationFailures`) plus an error-level log with the log name and byte offset. Better: persist a per-log corruption marker so the condition survives restart and can gate a "re-clone recommended" health state.
---
🤖 Investigated and filed by Claude (Fable) on Nathan's behalf
Contributor guide
Research direction
Start in resources/replayLogsGuards.ts at endIteratorOnCorruptFrame and trace how the RangeError latch currently logs and marks iteration done. Then inspect system_information, cluster_status, and the blobReplicationFailures pattern; done means corruption emits a structured signal with the log name and byte offset and is visible to operators.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- nodejs, typescript
- Domain
- databases, distributed-systems, observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100