HarperFast / HarperFast/harper

Two-phase deploy: package deploys are resolved per node, so the stage barrier cannot guarantee every node staged the same bytes

Open
#2,295 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 6h
Merged PRs (30d)
200

Description

Split out of #1849, which adds two-phase (stage → activate) component deploys.

For a `package:` deploy the origin replicates the package *identifier*, not bytes. Confirmed in that
branch: peers wait on the deployment row with `requirePayload: !spec.package`
(`components/operations.js`), so a package deploy deliberately has no shared payload and every node
independently resolves and packs the reference.

## The failure

A mutable reference can resolve differently per node:

- a moving tag such as `latest`
- a semver range
- a git branch

Each node then stages *a* component and reports success, so the barrier passes and activation
proceeds — while the nodes are running different code. The barrier confirms "everyone staged
something", not "everyone staged the same thing".

This is silent: every stage succeeds, the deployment rows all look healthy, and nothing compares what
was produced.

DESIGN.md is currently accurate but narrow — it scopes the barrier's guarantee to "fetch + install,
not load" and makes no claim about byte equality. So this is a missing guarantee rather than a
contradiction of documented intent.

## What a fix requires

Resolve once, on the origin:

1. Origin resolves and packs the package, as it already does for its own stage.
2. Persist the exact tarball plus a digest on the deployment row.
3. Peers stage *those* bytes and verify the digest instead of resolving independently.

Two consequences worth deciding deliberately:

- It changes what the barrier guarantees — from "each node installed something" to "every node
installed these bytes".
- It makes a `package:` deploy carry a payload it currently does not, with the storage and payload
retention cost that implies (`deployment_payloadRetention_maxCount` then applies to package
deploys too).

## Notes

- A pinned exact version resolves deterministically, so the divergence needs a mutable reference —
that is what keeps this contained relative to #2294.
- Exposure today is zero: the two-phase protocol is unshipped.

Contributor guide

Open the contributing guide

Research direction

Start with components/operations.js and DESIGN.md, then trace the deployment row and stage barrier described in the issue. Determine how origin package resolution, payload persistence and retention, peer staging, and digest verification fit the existing two-phase flow. Done means mutable package references produce one persisted payload and digest that all peers stage and verify, with the changed barrier guarantee documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.