CircleCI-Public / CircleCI-Public/node-orb
Re-examine disabling `~/.npm` caching for `npm ci` (PR #228)
- Dominant language
- Shell
- Stars
- 57
- Forks
- 76
- PR merge metrics
- No merged PRs in 30d
Description
## Summary
PR #228 ("Fix: nocache npm ci", shipped in v6.1.0) made `node/install-packages` skip **both** `restore_cache` and `save_cache` whenever the default `npm ci` path is used, on the rationale that *"the command doesn't benefit from cache."* That rationale is repeated in the parameter docs.
I'd like to re-open this, because:
1. The premise ("`npm ci` doesn't benefit from cache") is **incorrect as stated** — `npm ci` does reuse the global `~/.npm` cache.
2. PR #228 and issue #202 **never actually examined the tradeoffs** — only build-minutes and CircleCI cache-storage cost were weighed; bandwidth/egress, private-registry latency & rate-limits, and offline/repeatability were not.
3. The `cache-path` parameter documentation is now **factually wrong**.
I'm not arguing the default is necessarily wrong — for fast public registries it may well be right. I'm arguing the decision deserves a documented cost/benefit analysis and probably a first-class opt-in, rather than an undocumented workaround.
## 1. `npm ci` *does* use the global cache (evidence)
I ran a controlled experiment in a clean Docker container (per-package named cache volume to guarantee cold vs warm isolation; mitmproxy logging every registry request; `strace` to prove zero sockets when offline). Node 24 / npm 11, `lockfileVersion: 3`. Public sample `lodash`, private sample from a GitHub Packages registry (131-package tree).
| Scenario | Cache | Network | Tarballs downloaded | Registry calls |
|---|---|---|---|---|
| 1st `npm ci` | cold (empty `~/.npm`) | online | 1 / 126 | many |
| 2nd `npm ci` | warm | online | **0 / 0** | **0 / 0** |
| `npm ci --prefer-offline` | warm | online | **0 / 0** | **0 / 0** |
| `npm ci --offline` | warm | **disabled** | **0 / 0** | **0 / 0** (succeeds) |
(values are `public / private`)
A warm `npm ci` downloads **nothing** and makes **zero** registry calls, and a warm `npm ci --offline` succeeds with the network physically cut off (strace confirms zero `connect()` attempts). So the cache *is* used; it eliminates all tarball downloads.
The likely source of the "doesn't benefit" belief: with a warm cache, **wall-clock time barely changes** (e.g. 12.7s → 11.9s for the 131-pkg install), because `npm ci` is dominated by unpacking/linking, not downloading. The cache saves **bandwidth**, not necessarily time — those are different benefits and shouldn't be conflated.
## 2. Tradeoffs PR #228 didn't weigh
The cache-vs-no-cache decision for `npm ci` is multi-dimensional; #202/#228 only considered the last two:
- **Bandwidth / registry egress** — cold runs re-download every tarball, every build.
- **Private-registry latency & rate limits** — many private registries (Artifactory, GitHub Packages, etc.) are slower and rate-limited; cold installs hit them hard.
- **Offline / repeatability resilience** — a warm cache makes installs robust to registry blips.
- **CircleCI cache transfer + storage cost** — `restore_cache`/`save_cache` upload/download a cache blob to CircleCI storage (network + billed storage). If it doesn't cut wall-time, it's net cost. *(This is the legitimate point from #202.)*
The right default genuinely depends on these. Disabling unconditionally optimizes only for the last bullet and silently discards the first three — with no flag to opt back in (see §4).
## 3. The "registry is fast" assumption is weak (esp. private registries)
Worth noting because it shapes the cost side: a metadata "list"/packument returns **all** versions' metadata, and the full packument includes per-version README + full package.json — tens of MB for long-lived packages. npm mitigates this by requesting **abbreviated** metadata by default (`Accept: application/vnd.npm.install-v1+json`). Measured:
- **npmjs.org honors it** — `lodash` 247 KB → 70 KB; `@aws-sdk/client-sns` 2.85 MB → 1.58 MB.
- **GitHub Packages ignores it** — a private package's packument was **1,985,639 bytes with and without** the header (byte-identical), still carrying full per-version metadata incl. `readme`, across **892 versions**.
Caveat for honesty: this hits the **`npm install`/resolution** path, **not `npm ci`** (which with a v3 lockfile makes *zero* packument requests). So it's not a direct `ci` cost — but it demonstrates that "the registry is fast/cheap, just refetch" is a shaky blanket assumption, especially for the private registries this orb's users rely on.
## 4. Re-enabling the cache today requires an undocumented quirk
Because the skip conditions compare `override-ci-command` against the literal string `"npm-ci"` (hyphen) while the default command is `npm ci` (space), the only way to keep `npm ci` **and** re-enable `~/.npm` caching is:
```yaml
- node/install-packages:
override-ci-command: npm ci # space, so != the "npm-ci" sentinel -> caching turns back on
```
This works but relies on a sentinel mismatch, not a documented feature. A clear parameter (e.g. `cache-npm-ci: true`) would be far better.
## 5. Documentation bug
The `cache-path` parameter description still says:
> By default, this orb will cache `~/.npm` for npm…
This is **false** since v6.1.0 — on the default `npm ci` path nothing is cached. The description also internally contradicts itself ("The cache will be ignored when using npm ci"). Please correct it regardless of the outcome above.
## Proposed
1. Add a documented opt-in to cache `~/.npm` for `npm ci` (e.g. `cache-npm-ci: true`), replacing the `override-ci-command: "npm ci"` quirk.
2. Fix the `cache-path` doc string to state the actual default behavior.
3. Optionally, document the tradeoff (bandwidth/private-registry vs CircleCI storage cost) so users can choose deliberately.
Happy to provide the full experiment harness / raw logs if useful.
Contributor guide
No contributing guide indexed for this repository
Research direction
Inspect the node/install-packages entry point, especially the default npm ci path and its override-ci-command sentinel checks, then review the cache-path parameter documentation. Compare the existing cache behavior with the proposed opt-in and update the documented behavior; done means the choice is explicit and the cache-path text is factually consistent.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- node.js
- Domain
- ci-cd, documentation
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100