HarperFast / HarperFast/harper-pro
Subscription-setup stagger is added on top of independent jitter draws, so the #446 spacing does not hold on stale-worker reassignment
- Dominant language
- JavaScript
- Stars
- 3
- Forks
- 0
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 80
Description
Found by a full cross-model re-read of [#800](https://github.com/HarperFast/harper-pro/pull/800) (branch `fix/replication-uniform-backoff`), recorded in that PR's description and deliberately not fixed there. **This is unmerged code** — if #800 addresses it before merge, close this.
## What is wrong
`createSubscribeSetupScheduler.schedule()` computes the armed delay as `delay = backoffDelay + staggerMs`, where `backoffDelay` comes from a `createBackoff` **owned per `(url, database)`**, each with its own RNG:
```js
schedule = { backoff: createBackoff({ initialMs, maxMs, minMs, random }) };
...
const backoffDelay = schedule.backoff.nextDelay();
const delay = backoffDelay + staggerMs;
```
The caller then adds the fixed ladder that is supposed to space the dials out:
```js
// Stagger the subscribe when reassigning (subscribeStagger set) so N databases on one peer
// don't dial N catchup connections simultaneously. See #446.
const staggerMs = subscribeStagger ? subscribeStagger.count++ * RECONNECT_STAGGER_MS : 0;
```
Adding a fixed ladder to **independent** jittered draws does not produce fixed spacing. Two adjacent databases drawing 399 ms and 200 ms fire at 399 ms and 250 ms — 149 ms apart where the ladder intends `RECONNECT_STAGGER_MS`; other pairs can land on the same tick, or invert order. So a stale-worker reassignment can burst several WebSocket/TLS setups at once, which is the concurrency #446 exists to bound.
## Why it is easy to miss
The sibling path does it correctly: the recovery sweep draws **one** jitter base and uses it as a common base, so the ladder survives decorrelation there. Only the stale-worker reassignment path adds the ladder to per-database draws. #800's summary is accurate about the sweep and simply silent about this path, which is what makes the gap easy to read past.
## The shape of a fix
Draw one jitter base per reassignment pass and add the ladder to that, mirroring the sweep. Note this is a real trade rather than a typo fix: a shared base reintroduces the cross-database correlation on one peer that the per-schedule draws were chosen to avoid, so it is worth deciding deliberately.
Refs #800, #446, #327
Contributor guide
Research direction
Read createSubscribeSetupScheduler.schedule() and its per-(url, database) createBackoff ownership, then compare it with the recovery sweep described in the issue. Verify that reassignment delays preserve RECONNECT_STAGGER_MS spacing and decide the shared-base correlation trade-off; done when the stale-worker path no longer permits burst or inverted dials.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js
- Domain
- backend, distributed-systems, networking, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 48/100