HarperFast / HarperFast/harper
Audit purge cannot be told apart from a blocked audit purge: both return transactions_deleted: 0 with no signal
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 200
Description
An audit purge that deletes nothing is indistinguishable from an audit purge that was *blocked* from
deleting anything: both return `{transactions_deleted: 0}` with HTTP 200 and no distinguishing log
line. rocksdb-js already exposes the signal that separates them; harper never reads it.
## Mechanism (harper, `origin/main`)
`dataLayer/harperBridge/ResourceBridge.ts:511-550` (`deleteTransactionLogsBefore`) and the deprecated
wrapper `dataLayer/delete.ts:95-113` return `{transactions_deleted: 0}` / `{entries_deleted: 0,
log_files_deleted: 0}` identically whether
- nothing was eligible for purge (retention cutoff older than every log), or
- files *were* past retention but retained because they have not flushed to RocksDB yet.
There is a single unconditional `harperLogger.info('Finished deleting audit logs before …')`
regardless of outcome, and no branch inspects any retention-gate signal.
`resources/RocksTransactionLogStore.ts:462-472` (`getStats()`) surfaces only `{logs, totalSize}`.
Meanwhile rocksdb-js implements the distinguishing gauge end to end:
`log.getStats().purge.retainedUnflushedFiles` — native (`src/binding/transaction_log/transaction_log.cpp:401`,
`transaction_log_store.cpp:621`), typed (`src/load-binding.ts:195`), documented ("useful for
diagnosing why logs are not being cleaned up… files past retention but kept (not yet flushed)"),
and covered by `test/transaction-log-stats.test.ts:111`. Nothing in harper's purge path calls it.
## Why file it separately
This is orthogonal to #1294 / PR #1406, which made the reported count truthful but changed neither
granularity nor observability of the retained-files case. A retention fix alone would leave the
operator unable to tell "nothing to do" from "blocked, retry later".
## Fix shape
Thread `retainedUnflushedFiles` (or an equivalent rotation-state gauge) through the purge result
object, and/or emit a distinguishing log line for the retained-files zero-case.
Contributor guide
Research direction
Start in dataLayer/harperBridge/ResourceBridge.ts:511-550 and dataLayer/delete.ts:95-113, then inspect resources/RocksTransactionLogStore.ts:462-472 and the rocksdb-js stats gauge. Use test/transaction-log-stats.test.ts:111 as the signal reference. Done means a zero-result purge distinguishes no eligible files from files retained because they are unflushed, through its result or a log line.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- node.js, typescript
- Domain
- databases, observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100