Intermittent: megarepo sync fails with 'Cannot find module @effect/cli'

Summary

We have seen an intermittent failure when running megarepo:sync / mr sync --all from a source-CLI checkout (Bun executing packages/@overeng/megarepo/bin/mr.ts). The process sometimes aborts early with a module resolution error for @effect/cli.

This is hard to reproduce on demand, but we have a concrete failure log and a deterministic induced reproduction when the worktree dependencies are missing/partially present.

Observed Error

Representative excerpt:

error: Cannot find module '@effect/cli' from '<megarepo-store>/.../effect-utils/refs/heads/main/packages/@overeng/megarepo/bin/mr.ts'
Bun v1.x

Context

• mr.ts imports @effect/cli as a bare specifier.
• packages/@overeng/megarepo/package.json declares @effect/cli as a direct dependency (devDependency).
• In megarepo workflows, repo checkouts often live under a cache/store directory (e.g. ~/.megarepo/.../refs/...).
• In source CLI mode, mr relies on node_modules being present in that worktree. If node_modules is missing or incomplete, Bun fails immediately before any CLI logic runs.

Investigation Notes

What we verified:

• The failure path is consistent with "source-CLI execution + missing/incomplete deps".
• In a controlled test where packages/@overeng/megarepo/node_modules is removed/hidden, bun bin/mr.ts sync --help fails deterministically with module-not-found.
• In a full cached worktree where deps were present, repeated runs (including concurrent stress) succeeded.

Hypothesis (most likely):

• The install/readiness signals around packages/@overeng/megarepo/node_modules can sometimes be stale or wrong (e.g. partial install, drift, or a cached task saying "installed" while critical modules are not actually resolvable), leading to an intermittent startup failure.

Options To Address (with trade-offs)

1. Add a preflight to mr startup (or the megarepo:sync wrapper) that asserts required modules are resolvable.

   • Example: verify require.resolve('@effect/cli') / Bun equivalent.
   • If missing: fail with a clear remediation message (or optionally trigger install).
   • Trade-offs: slightly more work on each run; but fast and makes failures actionable.

2. Make mr execution more hermetic (prefer a Nix-built/bundled CLI for sync).

   • Trade-offs: higher determinism, but less "live edit" friendliness and more build plumbing.

3. Change task caching/status checks (where applicable) to validate a real capability (module resolution) instead of only directory/hash markers.

   • Trade-offs: more accurate, but may increase install task frequency and cost.

4. Workflow-level mitigation: avoid hard-failing shell startup on sync failures; run sync explicitly.

   • Trade-offs: better UX, but can mask drift and move failures later.

If This Happens Again (data to capture)

• The exact command used (mr ... vs bun .../mr.ts ... vs nix run ...).
• Whether packages/@overeng/megarepo/node_modules/@effect/cli exists in the executing worktree.
• Bun version.

(Posted on behalf of the user by an AI assistant.)

Start with packages/@overeng/megarepo/bin/mr.ts and its package.json, then reproduce with bun bin/mr.ts sync --help after hiding packages/@overeng/megarepo/node_modules. Check how dependency readiness is determined in the megarepo workflow and whether @effect/cli is resolvable in the executing worktree. Done means the selected mitigation prevents or clearly diagnoses startup failure when dependencies are missing, with the affected sync path verified.