HarperFast / HarperFast/harper
Backport #2345 (build-aside component deploy) to 5.2 — an interrupted deploy leaves the component unloadable, and no released 5.2.x has the fix
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 200
Description
## What happened
On an internal two-node control-plane cluster running a released `5.2.x`, a
`deploy_component` left the node serving the **new** source tree with **no `node_modules`
at all**. Every bare import out of the component failed:
```
ResourceLoadError: Failed to load resource module .../components//resources.js:
Cannot find module '@harperfast/oauth'
...
Cannot find module 'semver' # different dependency, different source file
```
Both are direct dependencies in the component's own `package.json`, so this is not a
resolution quirk in one package — the dependency install did not happen. The component
answered HTTP 500 on every route until it was redeployed. It happened on one node, then
on the other an hour later on a separate, non-replicated deploy.
The same payload deployed successfully to each node on the immediate retry, so nothing
about the component's dependency set is at fault: `npm install --force --ignore-scripts`
against that exact `package.json` + lockfile resolves clean (1517 packages, exit 0).
## Why an interrupted install is unrecoverable before #2345
This is exactly the failure #2345 describes:
> The live tree was moved aside before the replacement existed … then the tarball was
> extracted into the emptied live path and `npm install` ran there … the live path held
> the **new** code before its dependencies were installed.
In the pre-#2345 `components/Application.ts`:
1. `extractApplication` does `rm(dirPath, { recursive: true, force: true })` on the **live**
directory and extracts into it. The working deployment, `node_modules` included, is
destroyed before anything replaces it.
2. `installApplication` runs `npm install` afterwards, in that same live path.
3. If step 2 does not complete, the node is left with new source and no dependencies —
and **nothing repairs it**:
- boot-time installation (`server/loadRootComponents.js` → `installApplications()`) only
walks root-config entries that carry `package:`, so a payload-deployed component never
gets a second chance at an install;
- `installApplication` returns early when `node_modules` merely *exists* — a presence
check, not a validity check — so a partial tree is sticky rather than repaired.
A restart does not clear it. Only a successful redeploy does.
## No released 5.2.x has the fix
Resolved each release's `core` submodule pin and checked it for the `#2345` functions:
| harper-pro release | core pin | build-aside deploy |
|---|---|---|
| v5.2.7 / v5.2.8 / v5.2.9 | `f883a3bf` / `23264ceb` / `1d06b9fd` | no |
| **v5.2.10, v5.2.11** (current stable) | `e91e607a` | **no** |
| v5.3.0-alpha.1 | `e68efe91` | yes |
| harper-pro `main` | `d6f4efae` | yes |
So every shipped 5.2.x, including current stable, still empties the live component
directory before it has a replacement.
## Ask
Backport #2345 — and whatever earlier steps of the #2315 sequence it depends on — to the
5.2 line; or state explicitly that component deploys on 5.2 are known to be non-atomic and
the remedy is upgrading to 5.3.
## Two things #2345 does not appear to change
Worth deciding separately, since they outlive the rewrite:
1. **`installApplication` still returns early when `node_modules` exists.** It is a
presence check. A tree left partial by a killed install is indistinguishable from a
complete one, so the repair path is "redeploy", never "install the missing half".
2. **A deploy whose response is lost is indistinguishable from one that succeeded.**
`replicateOperation` returns per-peer failures in `response.replicated` rather than
throwing, so when the response does not reach the caller — the connection here was reset
mid-deploy — a failed peer is reported nowhere at all. That is why a production
component stayed down with the deploy showing green; the caller's own exit-code handling
(a v4 CLI in our case) made it worse, but the result is unobservable from the API even
with a correct client.
## What is not established here
Why the dependency install did not complete on those two deploys is **not** root-caused —
that needs the server-side log from the deploy window, which was not captured before the
nodes were repaired. This issue is about the blast radius of that failure, not its trigger:
before #2345 any interruption in that window is fatal to the component and invisible to the
deployer.
Contributor guide
Research direction
Start with the pre-#2345 components/Application.ts flow, then compare the 5.2 core pin with #2345 and the earlier #2315 sequence. Read server/loadRootComponents.js and the installApplications/installApplication paths to understand the boot-time and partial-install behavior. Done means the applicable changes are backported to 5.2, or the issue clearly records that 5.2 component deploys remain non-atomic and require 5.3.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, nodejs, typescript
- Domain
- backend, devops, release
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100