intuit / intuit/infigraph

Watcher holds graph write lock for its whole lifetime, blocking readers and leaking memory

Open
#46 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
88
Forks
16
PR merge metrics
No merged PRs in 30d

Description

## Summary

`infigraph watch` holds the Kuzu/LadybugDB write lock (and an open DB connection) for the **entire lifetime** of the watcher process, not just during a reindex. The DB is single-writer, so an idle watcher — which is ~99% of the time — needlessly blocks other operations and pins memory.

## Symptoms

1. **Read queries fail while a watcher is running.** Running `infigraph query` / `get_file_deps` / etc. against a repo that has a live watcher fails with:
```
Error: graph is locked by another infigraph process (e.g. a running `infigraph watch`) -- not corrupted, so it was left untouched.
Caused by: failed to open kuzu db: IO exception: Could not set lock on file : .../.infigraph/graph
```
2. **Orphaned watchers accumulate and leak memory.** Each `index` auto-spawns a watcher; if they aren't cleaned up they pile up, each pinning ~100–425MB. On a constrained machine this starved RAM enough to OOM-kill (`cc ... signal: 9`) large link steps.

## Root cause

The watcher acquires and holds a write-capable connection / lock for its whole run, conflating **watcher liveness** with **lock ownership**. It only actually needs write access *during* a reindex (on a file-change event).

The current invariant (`CLAUDE.md`): "file-watching uses a lock file for cross-process dedup — anything that starts a watcher must hold it for the watcher's lifetime." That's fine for the *dedup lock file*, but it shouldn't extend to the *graph write lock*.

## Proposal

- Watcher keeps only the **dedup lock file** for its lifetime (cross-process "one watcher per repo" guarantee).
- Watcher opens a graph **write connection lazily, per reindex**, and drops it immediately after the write completes.
- Idle watchers then hold no graph lock → concurrent readers (`query`, `get_file_deps`, MCP read tools) work without contention, and idle memory drops.

### Care required (single-writer invariant)

- Two watchers on the same repo (should already be prevented by the dedup lock file).
- A watcher racing a manual `infigraph index` — needs the same on-demand acquire/backoff both sides.
- Reindex atomicity — acquire → delete-stale + bulk-insert + resolve → release must stay one critical section.

Relates to the deferred "DB write serialization daemon" idea.

_Found while working on unrelated fixes; filing so it isn't lost._

Contributor guide

Open the contributing guide

Research direction

Read CLAUDE.md, then trace the `infigraph watch` and `infigraph index` entry points, along with the `query` and `get_file_deps` commands that currently fail during watching. Verify that the dedup lock remains for the watcher lifetime, while graph write access is acquired only for each reindex and released afterward; readers must work while the watcher is idle.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.