HarperFast / HarperFast/harper

Audit purge cannot be told apart from a blocked audit purge: both return transactions_deleted: 0 with no signal

Open
#2,455 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 6h
Merged PRs (30d)
200

Description

An audit purge that deletes nothing is indistinguishable from an audit purge that was *blocked* from
deleting anything: both return `{transactions_deleted: 0}` with HTTP 200 and no distinguishing log
line. rocksdb-js already exposes the signal that separates them; harper never reads it.

## Mechanism (harper, `origin/main`)

`dataLayer/harperBridge/ResourceBridge.ts:511-550` (`deleteTransactionLogsBefore`) and the deprecated
wrapper `dataLayer/delete.ts:95-113` return `{transactions_deleted: 0}` / `{entries_deleted: 0,
log_files_deleted: 0}` identically whether

- nothing was eligible for purge (retention cutoff older than every log), or
- files *were* past retention but retained because they have not flushed to RocksDB yet.

There is a single unconditional `harperLogger.info('Finished deleting audit logs before …')`
regardless of outcome, and no branch inspects any retention-gate signal.
`resources/RocksTransactionLogStore.ts:462-472` (`getStats()`) surfaces only `{logs, totalSize}`.

Meanwhile rocksdb-js implements the distinguishing gauge end to end:
`log.getStats().purge.retainedUnflushedFiles` — native (`src/binding/transaction_log/transaction_log.cpp:401`,
`transaction_log_store.cpp:621`), typed (`src/load-binding.ts:195`), documented ("useful for
diagnosing why logs are not being cleaned up… files past retention but kept (not yet flushed)"),
and covered by `test/transaction-log-stats.test.ts:111`. Nothing in harper's purge path calls it.

## Why file it separately

This is orthogonal to #1294 / PR #1406, which made the reported count truthful but changed neither
granularity nor observability of the retained-files case. A retention fix alone would leave the
operator unable to tell "nothing to do" from "blocked, retry later".

## Fix shape

Thread `retainedUnflushedFiles` (or an equivalent rotation-state gauge) through the purge result
object, and/or emit a distinguishing log line for the retained-files zero-case.

Contributor guide

Open the contributing guide

Research direction

Start in dataLayer/harperBridge/ResourceBridge.ts:511-550 and dataLayer/delete.ts:95-113, then inspect resources/RocksTransactionLogStore.ts:462-472 and the rocksdb-js stats gauge. Use test/transaction-log-stats.test.ts:111 as the signal reference. Done means a zero-result purge distinguishes no eligible files from files retained because they are unflushed, through its result or a log line.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, typescript
Domain
databases, observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.