overengineeringstudio / overengineeringstudio/effect-utils

DistributedSemaphore wakeup depends on unreliable FileSystem.watch event tags

Open
#986 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area:effect system:utils type:chore
Dominant language
TypeScript
Stars
82
Forks
2
Avg merge
1d 8h
Merged PRs (30d)
121

Description

Problem

DistributedSemaphore's release-notification path branches on FileSystem.watch event tags, filtering to Update | Remove and ignoring Create:

packages/@overeng/utils/src/node/file-system-backing.ts:376-379

Genie target locks use this backing.

Watch event tags turn out not to be reliable. Probing Effect 3 directly: a file can be created, modified and deleted — with fs.exists confirming it is gone afterwards — while no Remove tag is ever emitted, reproducibly, even when the operation is re-driven. Event ordering, coalescing of rapid writes, and tag classification all vary run to run within a single major.

So any consumer that decides what happened from the tag is building on an unreliable signal.

Why this is not urgent

DistributedSemaphore always races the watch push stream against a 100ms polling acquire loop. A missed or misclassified tag therefore delays wakeup by at most one poll interval — it does not cause a correctness or liveness failure. This is a fragile optimisation, not a live defect.

That is also why it has never shown up: the polling fallback quietly absorbs it.

Why fix it anyway

The tag filter buys nothing that path filtering does not already buy, and it depends on a property the platform does not actually guarantee. Treating any event on a watched path as a wakeup is simpler, strictly more reliable, and removes a dependency on behaviour that varies between runs — and between Effect majors.

Other FileSystem.watch consumers already do this: genie watch mode and both notion-md watchers filter by path only and re-read. megarepo's StoreLock explicitly strips the notification path and polls. This site is the outlier.

Proposed change

Treat any event on a matching path as a wakeup; drop the Update | Remove tag filter.

Acceptance

  • no watch consumer branches on event tag classification
  • a test covers wakeup on a Create-tagged event
  • the polling fallback remains as the correctness guarantee, not the primary path

Context

Found while characterising FileSystem.watch behaviour for the Effect 4 migration (#925). Independent of that migration — it is a pre-existing property of Effect 3.

Posted on behalf of @schickling
field value
agent_name cl1-heron
agent_session_id 54711470-ae7e-4322-a3e2-129a2689a097
agent_tool Claude Code
agent_tool_version 2.1.220
agent_runtime Claude Code 2.1.220
agent_model claude-opus-5
runtime_profile /nix/store/xg6r3nkr7spjyg2gbl3zprqbsag3fsjc-coding-agent-runtime-profile/share/coding-agents/profile.json
skills_manifest /nix/store/2vkpfkkbc9hjmcwknzkv62v2i44an8mz-agent-skills-corpus/share/agent-skills/manifest.json
worktree effect-utils/schickling/2026-07-28-effect-4
machine dev3
tooling_profile dotfiles@unknown-dirty

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at packages/@overeng/utils/src/node/file-system-backing.ts:376-379 and trace the DistributedSemaphore release-notification path. Add coverage for a Create-tagged event, then verify that matching-path events wake the semaphore while the 100ms polling fallback remains; the acceptance checklist defines done.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.