HarperFast / HarperFast/harper
Blob GC: files not deleted on audit-log expiry when using RocksDB audit store
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 205
Description
## Summary
When a Blob-typed record is deleted via SQL and `auditRetention` is configured, the blob file on disk is never cleaned up if the audit store is backed by RocksDB (the default).
## Root cause
In `dist/resources/auditStore.js`, `scheduleAuditCleanup()` branches on audit store type:
```js
if (auditStore instanceof RocksTransactionLogStore) {
auditStore.rootStore.purgeLogs({ before: ... });
return; // <-- returns without blob file cleanup
}
// LMDB path: iterates audit entries and calls removeAuditEntry() which
// invokes deleteCallbacks to remove blob files
```
The RocksDB path calls `rootStore.purgeLogs()` and returns immediately. `removeAuditEntry()` — which is responsible for invoking blob-file delete callbacks — is never called. Additionally, `RocksTransactionLogStore.remove()` is a no-op with a TODO comment:
```js
async remove() {
// TODO: this function can likely be removed once the call to purgeLogs()
// is added in `resources/Table.ts`
}
```
## Effect
Blob files accumulate on disk indefinitely after the DB record is deleted, until the schema or table is dropped (which uses a direct filesystem operation, not the GC path).
## Confirmed via
Integration test `Blob lifecycle > blob file removed from filesystem after auditRetention expires` in `integrationTests/apiTests/blob.test.mjs` (PR #648). The test waits up to 35 seconds but blob files never disappear. `drop_schema` and `drop_table` tests that use direct filesystem operations pass fine.
## Workaround
Filesystem cleanup works when the schema or table is explicitly dropped (`drop_schema`, `drop_table` with `drop_records: true` — though the latter also appears incomplete). The audit-log GC path is the only broken case.
Related: #38 (Support running on Bun)
Contributor guide
Assessment
This issue has not been assessed yet.