HarperFast / HarperFast/harper-pro
Bound the bulk-copy walk's lookahead past outstanding blob transfers
- Dominant language
- JavaScript
- Stars
- 3
- Forks
- 0
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 80
Description
Follow-up to #699 (whose fixes — copy-cursor banking + in-place blob repair — treat the symptoms; this bounds the disease's blast radius) and a sibling to the in-band per-blob re-request idea from #699's suggestions.
## Proposal
Bound how far the bulk-copy send walk may run ahead of the oldest outstanding blob transfer: after sending a record whose blob is still in flight, allow only a small number of further records (e.g. a few) before pausing the walk until that blob completes. Sender-side only, no wire change (cross-version safe), behind a config key (e.g. `replication.copyBlobLookahead`, `0` = today's unbounded behavior).
## Why
Every deep failure in this family is downstream of the walk running arbitrarily far ahead of unsettled blobs:
- the #426 onCommit-await circular deadlock,
- the consumer-less PassThrough HWM deadlock (copyModeBlobDeadlock),
- unbounded receiver-side unsettled-blob state (the #699 watermark's memory bound exists to contain exactly this),
- large per-cycle re-walk loss on gap reconnects (bounded to the lookahead window instead).
With a small lookahead the durable copy cursor trails the walk head by at most N records, a blob fault costs at most N records per reconnect cycle, and receiver buffering is bounded by construction — a structural guarantee rather than per-bug patches. The stated goal of streaming ahead is keeping the connection saturated, and during a blob-dense copy the blob chunks are the bulk bytes anyway; a record with blobs is usually surrounded by other records with blobs.
## Costs to evaluate (benchmark before defaulting on)
- Head-of-line blocking: one slow blob stalls the walk for its duration, where today other transfers proceed concurrently (bounded by `MAX_OUTSTANDING_BLOBS_BEING_SENT`).
- Mixed tables: a stretch of blob-less records behind one slow blob would stall despite being free to stream.
- Suggested evaluation: blob-dense copy throughput and convergence time at N ∈ {0 (today), 2, 8, 32} on the copyGapCursorBanking harness plus a mixed-table variant.
Origin: Kris's suggestion while reviewing the #699 fix pair (harper-pro#701 / harper#2177).
Contributor guide
Research direction
Start with the copyGapCursorBanking harness and identify the sender-side bulk-copy walk and blob-transfer state it exercises. Evaluate lookahead values N ∈ {0, 2, 8, 32}, including a mixed-table variant, and compare blob-dense throughput and convergence time. Done means the configured bound works without a wire change and the benchmark results support whether a nonzero default is appropriate.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js
- Domain
- databases, distributed-systems, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100