overengineeringstudio / overengineeringstudio/effect-utils
CI: cold-install pnpm-state flake on Namespace runners amplified by success()-gated cache-save
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 82
- Forks
- 2
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 121
Description
Downstream consumers on Namespace cloud runners intermittently fail CI on lock-changing PRs with a cold-install node_modules/.bin materialization failure (Executable not found in $PATH: vitest/madge), preceded by a determinate-nix daemon self-test error (cannot connect to socket at /nix/var/nix/daemon-socket/socket). Observed on livestorejs/livestore#1397 (effect-utils bump). Same commit reruns rotate which jobs pass/fail → confirmed runner-health flake, not code.
Why it sticks (amplifier)
The generated per-job Save pnpm state step is gated:
if: ${{ success() && steps.restore-pnpm-state.outputs.cache-hit != 'true' }}
On a lock change every job cold-misses the pnpm-state cache. success() = whole-job success, so any job whose install completes but whose later step flakes (or whose daemon self-test fails) never seeds the cache → subsequent reruns keep cold-missing → the flake-prone cold path runs every time until a fully-green run happens by luck.
Two improvements (durable, careful — not urgent)
- Safe cache seeding: seed the pnpm-state cache when the install itself verifiably completed (node_modules/.bin populated, .modules.yaml present, install-state/projection hashes written) even if a later job step fails — but NOT when the install is incomplete (avoid caching a broken
links/). This raises seeding probability → fewer flake-prone cold installs on rerun, without caching corruption. Requires a discrete verify-install step to gate on instead of whole-jobsuccess(). - Daemon/setup retry robustness: the existing
run-with-nix-gc-race-retry.shwraps task runs, not the Nix-daemon self-test / devenv setup phase. Wrap (or retry) the daemon-setup/install phase so a transientdaemon-socketfailure retries instead of failing the whole job.
Root cause of the flake itself is Namespace/determinate-nix daemon availability (external infra); these two changes reduce its blast radius on downstream CI. Refs: pnpm.nix cold-path ~L455-475 / L575-632; genie CI generator Save pnpm state step. Filed from the livestore#1397 investigation; relates to the effect-utils consumption/prevention story (frequent-bump + canary).
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with pnpm.nix around the referenced cold-path lines and the genie CI generator's Save pnpm state step; read how the current install and cache-hit conditions are produced. Inspect run-with-nix-gc-race-retry.sh and the devenv setup path for the daemon self-test. Done means incomplete installs cannot seed pnpm state, completed installs can seed it after later failures, and transient daemon-setup failures retry.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, shell, typescript
- Domain
- build-system, ci-cd, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100