HarperFast / HarperFast/harper-pro
Interrupted blob copy leaves superseded truncated stub files that nothing reclaims
- Dominant language
- JavaScript
- Stars
- 3
- Forks
- 0
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 80
Description
## What happens
Interrupting a node mid blob-copy (SIGKILL during replication blob transfer) leaves structurally-incomplete blob **stub files** on disk — 24 truncated placeholders in one measured run. Each is superseded by a correctly re-sent complete file on reconnect, so there is **no data-integrity impact** — but the stubs are never reclaimed by anything observed, so sustained restart churn is a monotonic disk-space leak.
## Mechanism
`core/resources/blob.ts`'s blob write path streams directly to the final path with no temp-file+rename step, so a kill mid-write leaves a truncated file at the real blob path; the resend lands beside it rather than replacing it. The existing integration test documents this exactly: `integrationTests/cluster/blobCopyInterruptionIntegrity.test.mjs` (~line 534) — "superseded by a correctly-resent complete file — no data-integrity impact, but not cleaned up immediately (disk-space leak under sustained churn). Deliberately not asserted — see PR #612 honest note."
## Suggested fix shape
Either write-to-temp + rename (kills the stub class at the source), or teach the orphan sweep to recognize superseded stubs (size/checksum mismatch against the blob record) as reclaimable.
## Existing work checked
harper-pro#385 (OPEN) is a *correctness* issue — a live-referenced corrupt stub needing `repair_blob_data`; distinct from this finding's already-superseded, safe-to-delete stubs. #406 (CLOSED 2026-06-17) is the replication version-conflict-skip cleanup path in `replicationConnection.ts` — different mechanism, different code path. Nothing covers superseded-but-unreclaimed stubs from an interrupted copy.
## What this does not prove
The stub growth rate under production churn (one measured run); whether `cleanup_orphan_blobs` would reclaim them if run manually (untested against this stub shape).
---
*From dispatch QA finding qa-wave-2026072422, verified against harper-pro origin/main, 2026-08-29.*
Contributor guide
Research direction
Start with the blob write path in core/resources/blob.ts and run the scenario documented near line 534 of integrationTests/cluster/blobCopyInterruptionIntegrity.test.mjs. Then inspect cleanup_orphan_blobs and the relevant replication flow; done means an interrupted copy does not leave superseded truncated stubs unreclaimed, with coverage for the observed leak without weakening data-integrity checks.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js, typescript
- Domain
- backend, databases, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100