HarperFast / HarperFast/harper

Corrupt-entry audit read logs the entire entry buffer per entry, unbounded

Open
#2,143 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 6h
Merged PRs (30d)
200

Description

## Summary

`readAuditEntry`'s catch logs the full entry buffer on every decode failure: [v5.2.1 `resources/auditStore.ts#L651`](https://github.com/HarperFast/harper/blob/v5.2.1/resources/auditStore.ts#L651) — `harperLogger.error('Reading audit entry error', error, buffer)` — with no rate limit or once-latch. The replay path's own `getValue` catch deliberately drops errors for exactly this reason (`replayLogs.ts#L132`, its comment cites millions observed in prod), but this upstream site wasn't given the same treatment.

## Impact

A log region with many corrupt entries (torn writes after an unclean shutdown) floods the logger with one full-buffer dump per entry during recovery/replication catch-up — precisely when the node is weakest. Log volume and allocation churn amplify the recovery pressure, and the flood buries the one line an operator actually needs.

## Suggested fix

Log the buffer at debug level only (or truncate to a short hex prefix), and rate-limit/latch the error-level line per log file — first occurrence + running count, matching how the replay summary already reports an aggregate skip count.

---
🤖 Investigated and filed by Claude (Fable) on Nathan's behalf

Contributor guide

Open the contributing guide

Research direction

Start in resources/auditStore.ts around line 651 and compare its readAuditEntry catch with the getValue catch in replayLogs.ts around line 132. Decide how the error-level report should be limited while avoiding full buffer logging. Done means corrupt-entry recovery no longer emits an unbounded full-buffer log per failure, while retaining an actionable aggregate or first-occurrence signal.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, typescript
Domain
backend, performance
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.