HarperFast / HarperFast/harper
Two-phase deploy: package deploys are resolved per node, so the stage barrier cannot guarantee every node staged the same bytes
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 200
Description
Split out of #1849, which adds two-phase (stage → activate) component deploys.
For a `package:` deploy the origin replicates the package *identifier*, not bytes. Confirmed in that
branch: peers wait on the deployment row with `requirePayload: !spec.package`
(`components/operations.js`), so a package deploy deliberately has no shared payload and every node
independently resolves and packs the reference.
## The failure
A mutable reference can resolve differently per node:
- a moving tag such as `latest`
- a semver range
- a git branch
Each node then stages *a* component and reports success, so the barrier passes and activation
proceeds — while the nodes are running different code. The barrier confirms "everyone staged
something", not "everyone staged the same thing".
This is silent: every stage succeeds, the deployment rows all look healthy, and nothing compares what
was produced.
DESIGN.md is currently accurate but narrow — it scopes the barrier's guarantee to "fetch + install,
not load" and makes no claim about byte equality. So this is a missing guarantee rather than a
contradiction of documented intent.
## What a fix requires
Resolve once, on the origin:
1. Origin resolves and packs the package, as it already does for its own stage.
2. Persist the exact tarball plus a digest on the deployment row.
3. Peers stage *those* bytes and verify the digest instead of resolving independently.
Two consequences worth deciding deliberately:
- It changes what the barrier guarantees — from "each node installed something" to "every node
installed these bytes".
- It makes a `package:` deploy carry a payload it currently does not, with the storage and payload
retention cost that implies (`deployment_payloadRetention_maxCount` then applies to package
deploys too).
## Notes
- A pinned exact version resolves deterministically, so the divergence needs a mutable reference —
that is what keeps this contained relative to #2294.
- Exposure today is zero: the two-phase protocol is unshipped.
Contributor guide
Research direction
Start with components/operations.js and DESIGN.md, then trace the deployment row and stage barrier described in the issue. Determine how origin package resolution, payload persistence and retention, peer staging, and digest verification fit the existing two-phase flow. Done means mutable package references produce one persisted payload and digest that all peers stage and verify, with the changed barrier guarantee documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js
- Domain
- backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100