dmtrKovalenko / dmtrKovalenko/fff

[Bug]: pi-fff external-path grep can permanently block the Pi process

Open
#843 3 comments 0 reactions 0 assignees View on GitHub
triaged
Dominant language
Rust
Stars
10.7k
Forks
446
Avg merge
1d 21h
Merged PRs (30d)
39

Description

### Frontend

Other / multiple — `@ff-labs/pi-fff` using `@ff-labs/fff-node`.

### Environment

- `@ff-labs/pi-fff`: reproduced from the 0.10.5 code path; the relevant external-finder path still appears present in 0.10.6
- Pi: 0.84.4
- Node: 26.0.0
- macOS 27.0

### Reproduction

Run an external absolute-path grep against a package under an otherwise broad indexed tree:

```text
ffgrep /allowRanges/ in ~/.pi/agent/npm/node_modules/pi-web-access/ limit 20
```

Intermittently, Pi remains on `Working...` indefinitely. The tool-level 10-second grep budget and the auxiliary finder startup timeout do not recover the process.

The target itself is small and the same search normally completes quickly, so this is not simply a slow full-tree scan.

### Expected behavior

The search should return matches, `No matches found`, or a bounded actionable timeout without blocking Pi's main JavaScript thread.

### Actual behavior

A synchronous native create/index/grep call can stop returning. Because the event loop is blocked inside FFI, an outer `Promise.race`/timer cannot fire and the whole Pi process remains stuck.

### Investigation

Instrumentation of the pi-fff auxiliary finder lifecycle showed this sequence in the reproduced case:

1. A broader auxiliary finder exceeded its scan/startup wait.
2. Native scan/warmup continued after the JavaScript-side wait timed out.
3. The finder remained reusable while the global creation queue was released.
4. A later external-root grep entered native FFI while the earlier native work was still active.
5. Once the synchronous FFI call stopped returning, the JavaScript timeout could not interrupt it.

There may also be an interaction with:

- #711 / #713 — reusing a broader finder can exclude an explicitly requested `node_modules` subtree
- #826 / #827 — zero-match searches can skip the configured time budget
- #746 / #750 — related broad-path initialization hangs and same-root deduplication

Those fixes improve routing/budget behavior, but a JavaScript timer still cannot terminate a native synchronous call that has already wedged.

### Locally verified mitigation

As a local diagnostic/fix, we moved **external-path grep only** behind a bounded child-process lifecycle:

- exact-root workers
- same-root startup deduplication
- serialized native finder creation across roots
- bounded hot-worker pool
- startup/request/abort/shutdown all terminate the child process when necessary
- workspace grep and fffind remain unchanged

Regression coverage includes cold/hot search, matches and zero matches, explicit `node_modules` roots, startup timeout, grep timeout, abort, concurrent roots, and recovery after termination.

With that isolation, the original target returns 38 matches in about 1.6 seconds, and a deliberately wedged native worker can be killed without freezing Pi.

I am filing the issue first because the preferred upstream fix may be either:

1. native cancellation/time-budget enforcement in the SDK/Rust owner, or
2. process/thread isolation in pi-fff for external-path operations.

I can submit the isolated pi-fff implementation if that direction is useful.

### Logs

No useful log is emitted once the synchronous FFI call blocks; the JavaScript process cannot progress far enough to report the timeout.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the external absolute-path grep with `ffgrep /allowRanges/ in ~/.pi/agent/npm/node_modules/pi-web-access/ limit 20`, then inspect the pi-fff external-path grep entry point and its fff-node synchronous FFI call. Done means the operation has native cancellation or process isolation so startup, grep, abort, and shutdown are bounded without blocking Pi, with recovery verified for matches, zero matches, concurrent roots, and explicit node_modules roots.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, rust
Domain
search
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.