overengineeringstudio / overengineeringstudio/effect-utils

devenv: genie:run fails when pnpm:install runs concurrently (status check lacks flock)

Open Beginner friendly
#545 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area:nix origin:agent type:agent-tooling type:bug
Dominant language
TypeScript
Stars
82
Forks
2
Avg merge
1d 8h
Merged PRs (30d)
121

Description

Problem

genie:run intermittently fails with:

error: Cannot find module '@effect/cli' from '.../packages/@overeng/genie/bin/genie.tsx'
Bun v1.3.10 (macOS arm64)

This happens during pre-commit hooks or when multiple devenv processes enter simultaneously.

Root cause

The pnpm:install exec script acquires an flock (exec 200>"$lockfile"; flock -w 600 200), but the status script does NOT. This creates a race window:

  1. Process A (e.g. shell entry) runs pnpm:install exec → acquires flock, starts installing
  2. Process B (e.g. pre-commit hook) runs pnpm:install status → hash file exists from prior install, node_modules/ dir exists (partially constructed), health check passes (only checks depth 1-2 symlinks) → status returns 0 → task is "Cached"
  3. Process B's genie:run fires immediately → Bun tries to resolve @effect/cli through partially-linked pnpm store symlinks → fails

The node_modules health check in the status script (check_node_modules_links_healthy) only checks symlinks at depth 1-2 in node_modules directories, so a missing transitive dependency like @effect/cli inside packages/@overeng/genie/node_modules/ is not detected.

Reproduction

Reliably reproducible using devenv tasks:

# 1. Ensure clean baseline (creates hash file + populates node_modules)
devenv tasks run pnpm:install --mode before --no-tui

# 2. Break a specific transitive dependency symlink
rm -rf packages/@overeng/genie/node_modules/@effect/cli

# 3. Race: start install (acquires flock), then immediately run genie
#    genie:run's pnpm:install dep status check passes (hash matches,
#    health check misses the broken nested link) → genie runs against
#    partially-linked node_modules
devenv tasks run pnpm:install --mode before --no-tui &
sleep 0.5
devenv tasks run genie:run --mode before --no-tui
# → genie:run fails with "Cannot find module '@effect/cli'"

The real-world trigger is concurrent shell entry + pre-commit:

# Same setup, but simulating the actual scenario
rm -rf packages/@overeng/genie/node_modules/@effect/cli
devenv tasks run pnpm:install --mode before --no-tui &
sleep 0.3
devenv tasks run check:quick --mode before --no-tui
# → genie:run fails, ts:check dependency-fails, check:quick dependency-fails

Options

A. Flock the status check too

Add the same flock to pnpm:install's status script. If an install is in progress, status blocks until it finishes, then correctly reports "already done." Downstream tasks never see partially-linked node_modules.

status = ''
  lockfile="${cacheRoot}/pnpm-install.lock"
  exec 200>"$lockfile"
  ${flock} -w 600 200 || exit 1
  # ... existing hash/health checks ...
'';

Pro: Simple, correct, no behavior change when no concurrent install.
Con: Status check blocks if install is running — slightly slower pre-commit start when shell entry is installing concurrently.

B. Non-blocking try-lock in status
exec 200>"$lockfile"
${flock} -n 200 || exit 1  # fail immediately if locked → task runner re-runs exec

Pro: Fast fail, status returns "needs install" immediately.
Con: Task runner will try to run exec, which will then block on the flock anyway — same end result as A but with an extra task-runner round-trip.

C. Sentinel file

Have exec create a sentinel before install and remove it after. Status fails fast if sentinel exists.

Pro: Non-blocking, no flock in status path.
Con: Stale sentinel if process is killed mid-install (needs cleanup logic).

Recommendation

Option A is the most principled. The status check should reflect the true state of node_modules, and during a concurrent install that state is "in flux." Blocking is correct — it means "install is not done yet."


Filed by an AI assistant on behalf of @schickling

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Locate the devenv task configuration containing the pnpm:install status script and compare it with the existing exec script's lock handling. Reproduce the documented concurrent install and genie:run commands, then verify that the status check waits for the lock and no longer allows a partially linked dependency to reach genie:run.

Written by the indexing model from the issue text.

Assessment

Tech stack
bun, typescript
Domain
build-system, tooling
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.