HarperFast / HarperFast/harper

deploy_component: surface mid-deploy progress from each peer back to the origin

Open
#885 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 6h
Merged PRs (30d)
200

Description

## Summary

Right now, the origin's `hdb_deployment.peer_results[i]` only gets populated when a peer's deploy *terminates* — via the per-peer `onPeerResult` callback that fires on `sendOperationToNode` settlement. Between dispatch and settlement, peers are dark from the origin's perspective: no phase, no install output, no indication of how far along they are.

For local-CLI use that's mostly fine (the operator can see their own node's progress). For Studio and any human watching a multi-node fleet deploy, it's a significant gap — a deploy that takes 3 minutes on a peer shows the row stuck at `replicating` with no `peer_results` entry until the very end.

## Proposal

Peers write their own phase / event_log progress into the replicated `hdb_deployment` row's `peer_results[i]` as they go. Specifically, the peer's `deploy_component` handler, when running on the peer side (i.e. `isReplicatedExecution && _deploymentId`), looks up the row by `deployment_id`, locates its own `peer_results` entry, and updates `progress: {phase, status, started_at, last_event_at}` on each phase transition. The row replicates back to the origin via the existing table-replication channel — no new RPC, no peer-progress message type alongside `BLOB_CHUNK`.

Concretely:

```ts
// Roughly, on the peer side, inside deployComponent
const ownEntry = peer_results.find((p) => p.node === thisNodeName) ?? { node: thisNodeName };
ownEntry.progress = { phase: 'extracting', status: 'in_progress', started_at: now };
await table.put(row);
```

The recorder pattern from the origin already supports coalesced flushes; this is the same shape on the peer side with a much smaller scope (just the one `peer_results[i]` entry, not the full event_log).

## Open questions

1. **Granularity.** Do we mirror the full origin event_log to the peer's `peer_results[i].event_log`, or just phase transitions + a final install summary? Full mirror is verbose (chatty installs explode replication traffic for a fleet); summary-only is enough for human consumption. I'd default to phase transitions + bounded install summary (~50 events).
2. **Restart phase.** Origin currently doesn't track its *own* restart phase in `peer_results` (because it's not a peer to itself). For a fully observable deploy we'd want some equivalent row attribute for the origin's restart — likely just a `restart_status` on the row, separate from `peer_results`.
3. **Race vs. blob fetch.** Peer can only write to its `peer_results[i]` once the row exists locally, which happens after table replication delivers it. The blob fetch already gates on `awaitDeploymentRow`; peer progress reporting starts after that succeeds.

## Scope / cost

Small. The recorder write pattern is established; we're adding a peer-side variant that only touches one entry of `peer_results`. The hardest part is making the upsert idempotent under origin's `recordPeer` re-applying terminal results — but `recordPeer` already does upsert-by-node-name and treats `progress` as opaque, so a terminal `{status: 'success'}` from origin's `replicateOperation` callback simply replaces whatever `progress` the peer wrote. That's the desired semantic.

## Acceptance

- Studio (or any `get_deployment` poller) sees `peer_results[i].progress.phase` advance through `extracting → installing → loading → replicating → restarting → success` on each peer in near-real-time (sub-second after each transition).
- A multi-node test deploys a slow-installing component and asserts that mid-deploy, at least one peer's `peer_results[i].progress.phase` is observably advancing while origin is still in `replicating`.
- Aggregate `peer_results[i].status` semantics unchanged; terminal results from origin's `onPeerResult` callback still win.

🤖 Generated by Claude

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.