overengineeringstudio / overengineeringstudio/effect-utils
devenv: genie:run fails when pnpm:install runs concurrently (status check lacks flock)
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 82
- Forks
- 2
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 121
Description
Problem
genie:run intermittently fails with:
error: Cannot find module '@effect/cli' from '.../packages/@overeng/genie/bin/genie.tsx'
Bun v1.3.10 (macOS arm64)
This happens during pre-commit hooks or when multiple devenv processes enter simultaneously.
Root cause
The pnpm:install exec script acquires an flock (exec 200>"$lockfile"; flock -w 600 200), but the status script does NOT. This creates a race window:
- Process A (e.g. shell entry) runs
pnpm:installexec → acquires flock, starts installing - Process B (e.g. pre-commit hook) runs
pnpm:installstatus → hash file exists from prior install,node_modules/dir exists (partially constructed), health check passes (only checks depth 1-2 symlinks) → status returns 0 → task is "Cached" - Process B's
genie:runfires immediately → Bun tries to resolve@effect/clithrough partially-linked pnpm store symlinks → fails
The node_modules health check in the status script (check_node_modules_links_healthy) only checks symlinks at depth 1-2 in node_modules directories, so a missing transitive dependency like @effect/cli inside packages/@overeng/genie/node_modules/ is not detected.
Reproduction
Reliably reproducible using devenv tasks:
# 1. Ensure clean baseline (creates hash file + populates node_modules)
devenv tasks run pnpm:install --mode before --no-tui
# 2. Break a specific transitive dependency symlink
rm -rf packages/@overeng/genie/node_modules/@effect/cli
# 3. Race: start install (acquires flock), then immediately run genie
# genie:run's pnpm:install dep status check passes (hash matches,
# health check misses the broken nested link) → genie runs against
# partially-linked node_modules
devenv tasks run pnpm:install --mode before --no-tui &
sleep 0.5
devenv tasks run genie:run --mode before --no-tui
# → genie:run fails with "Cannot find module '@effect/cli'"
The real-world trigger is concurrent shell entry + pre-commit:
# Same setup, but simulating the actual scenario
rm -rf packages/@overeng/genie/node_modules/@effect/cli
devenv tasks run pnpm:install --mode before --no-tui &
sleep 0.3
devenv tasks run check:quick --mode before --no-tui
# → genie:run fails, ts:check dependency-fails, check:quick dependency-fails
Options
A. Flock the status check too
Add the same flock to pnpm:install's status script. If an install is in progress, status blocks until it finishes, then correctly reports "already done." Downstream tasks never see partially-linked node_modules.
status = ''
lockfile="${cacheRoot}/pnpm-install.lock"
exec 200>"$lockfile"
${flock} -w 600 200 || exit 1
# ... existing hash/health checks ...
'';
Pro: Simple, correct, no behavior change when no concurrent install.
Con: Status check blocks if install is running — slightly slower pre-commit start when shell entry is installing concurrently.
B. Non-blocking try-lock in status
exec 200>"$lockfile"
${flock} -n 200 || exit 1 # fail immediately if locked → task runner re-runs exec
Pro: Fast fail, status returns "needs install" immediately.
Con: Task runner will try to run exec, which will then block on the flock anyway — same end result as A but with an extra task-runner round-trip.
C. Sentinel file
Have exec create a sentinel before install and remove it after. Status fails fast if sentinel exists.
Pro: Non-blocking, no flock in status path.
Con: Stale sentinel if process is killed mid-install (needs cleanup logic).
Recommendation
Option A is the most principled. The status check should reflect the true state of node_modules, and during a concurrent install that state is "in flux." Blocking is correct — it means "install is not done yet."
Filed by an AI assistant on behalf of @schickling
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Locate the devenv task configuration containing the pnpm:install status script and compare it with the existing exec script's lock handling. Reproduce the documented concurrent install and genie:run commands, then verify that the status check waits for the lock and no longer allows a partially linked dependency to reach genie:run.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- bun, typescript
- Domain
- build-system, tooling
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100