HarperFast / HarperFast/harper
Docker image still ships the react-native-fs subtree unpinned (residual from #1960/#2042)
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 200
Description
## Summary
#1960 (via #2042) made the Docker image install Harper by extracting the packed tarball and running `npm install` in place, so it reads `npm-shrinkwrap.json` off disk instead of re-resolving fresh. That fixed version pinning (the urgent motivator: `@harperfast/rocksdb-js` 2.6.0/2.6.1, published 2026-07-31, redefined a store option in a way that breaks every table open — the unpatched Dockerfile resolved to it, the fixed one resolves to the pinned 2.5.0).
It did **not** deliver the other half #1960 originally hoped for: excluding the `react-native-fs` subtree (`react-native`, `metro`, `hermes`, `@babel/*`, etc. — #1937) from the image.
## Root cause
`build-tools/prune-shrinkwrap-react-native.mjs` prunes the react-native subtree from the *shrinkwrap* that ships in the published tarball. It does not, and cannot by itself, touch the packed `package.json` — `alasql`'s own manifest inside the tarball still declares `react-native-fs` as an `optionalDependency`.
A registry install of harper as *someone else's* dependency never notices: npm treats the bundled shrinkwrap as fully authoritative for a shrinkwrapped package and doesn't re-derive anything from nested manifests.
The Docker image's install (since #2042) is different: `npm install` inside the extracted tarball treats it as the *root* project, and reconciles that project's own `package.json` against the lockfile. Since `package.json` still declares the optional edge and the lockfile doesn't have it, `npm install` re-adds the whole pruned subtree to satisfy it — confirmed empirically.
## Why this matters beyond image size
Initially this looked like "just" ~140MB of dead code sitting behind an `isReactNative` guard that's unreachable under Node. It's sharper than that: `@endo/static-module-record` — a genuine **production** dependency, used by the SES sandbox (`security/jsLoader.ts`) to parse and transform application source — depends on `@babel/parser`, `@babel/traverse`, and `@babel/types`, the same packages the react-native chain re-adds unpinned. When npm reconciles the two demands, the shared copy can resolve above the shrinkwrap's pin for those packages. Confirmed the mechanism; whether it's live today wasn't separately confirmed (npm would normally keep the locked entry since the caret ranges are satisfied), but it means this isn't purely a bloat issue — it's version-drift risk in a security-relevant parser.
## What was tried and rejected (#2042)
- **`npm ci`**, after also stripping `devDependencies` from the extracted `package.json` (needed to pass `npm ci`'s root sync check): still refuses, with `Missing: react-native-fs@2.20.0 from lock file` plus ~30 more entries. `npm ci` is strict enough to refuse re-adding a pruned subtree rather than silently reconciling it — closer to correct than plain `npm install`, but it won't proceed until the manifest and lock fully agree.
- **`npm install --omit=optional`** (tree-wide): does drop the react-native subtree, but *also* drops `@harperfast/rocksdb-js`'s per-platform native-binary `optionalDependencies` — confirmed the `linux-x64-glibc` binding package is absent, which would silently break every table open. Rejected.
## The remaining fix
`npm ci` against a package.json where `alasql`'s **own** packed manifest has also had the `react-native-fs` optionalDependency edge removed (not harper's root `package.json` — a third-party dependency's manifest, patched post-pack, inside our own published tarball). That's a materially bigger and more unusual change to the published npm artifact than the `devDependencies` strip #2042 already did (which only affects the Docker image's ephemeral extracted copy, never the published tarball).
Open questions for whoever picks this up:
- Where does the alasql-manifest patch belong — a new `build-tools/` script alongside the existing prune scripts, run at pack time? Only applied for the Docker image's extracted copy, or for the published tarball too (a bigger decision — it would change what registry consumers receive)?
- Does patching a third-party dependency's manifest inside our own package have any downstream effect on tooling that reads package manifests (license scanners, `npm ls`, audit tools)?
- Is `npm ci` viable end-to-end once this lands, or does something else in the tree also drift from the lock?
## References
- #1960 (the pinning fix this is the residual half of)
- #2042 (the PR that landed the pinning fix and documented this gap in `DESIGN.md` and `dependencies.md`)
- #1937 (original react-native-fs shrinkwrap prune)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Contributor guide
Research direction
Start by reading build-tools/prune-shrinkwrap-react-native.mjs and the Docker image changes from #2042, then compare the packed package.json with npm-shrinkwrap.json. Test the proposed manifest handling with npm ci in the extracted tarball, checking that the react-native subtree stays absent while the platform-specific @harperfast/rocksdb-js package remains present. Done means the packaging scope is decided and the Docker install no longer re-adds the pruned dependency tree.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, javascript, nodejs, react-native
- Domain
- build-system, devops
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100