activeloopai / activeloopai/hivemind

Recall UserPromptSubmit hook hangs (2s timeout) when embed-daemon.js launcher is missing from embed-deps despite embeddings enabled

Aberta
#296 1 comentário 0 reações 0 responsáveis Ver no GitHub
Linguagem predominante
TypeScript
Estrelas
1.6k
Forks
107
Merge médio
17h 30min
PRs com merge (30d)
6

Descrição

## Symptom

Every prompt in Claude Code fails with:

```
UserPromptSubmit hook timed out after 2s — output discarded. Raise the hook's "timeout" to allow more time.
```

It is **persistent across fresh sessions** and unaffected by anything in the user's `settings.json` (the offending hook lives in the plugin's own `hooks.json`). Net effect: proactive recall is **never** injected — it always times out and is discarded.

## Environment

- hivemind `0.7.116` (Claude Code plugin)
- Node `v25.8.1`, macOS `15.6.1` (arm64)
- `~/.deeplake/config.json` → `{ "embeddings": { "enabled": true } }`
- Embed deps installed: `~/.hivemind/embed-deps/node_modules` present (508 MB, `@huggingface/transformers`)

## Root cause

The embedding **daemon launcher script is missing** from the shared deps dir, even though `node_modules` is fully installed and `embeddings.enabled = true`:

```
$ ls ~/.hivemind/embed-deps/
node_modules/ package.json # <-- embed-daemon.js is absent
$ hivemind embeddings status
Daemon: (not present)
```

`SHARED_DAEMON_PATH` = `~/.hivemind/embed-deps/embed-daemon.js` (`src/embeddings/standalone-embed-client.ts:39`). `hivemind embeddings install` is what deposits it (`src/cli/embeddings.ts:123-131`, "Always (re)deposit the canonical embed-daemon.js"), copying from `pkgRoot()/embeddings/embed-daemon.js`. On this machine that file never landed — most likely lost across a plugin version bump (the deps dir survived, the launcher did not).

The failure then cascades through the recall hook:

1. `recall.js` runs as a **synchronous** `UserPromptSubmit` hook with `"timeout": 2` (`hooks/hooks.json`). Its internal budget `RECALL_BUDGET_MS` defaults to 1500 ms (`src/hooks/recall.ts:63`), designed to finish under the 2 s cap.
2. With `embeddings.enabled` and `HIVEMIND_SEMANTIC_SEARCH` unset, it takes the semantic path and calls `trySpawnDaemon(SHARED_DAEMON_PATH)` (`src/embeddings/standalone-embed-client.ts:178`).
3. The daemon entry file doesn't exist, so the spawn produces no listening socket. `waitForSocket()` (`:259`) loops until its deadline instead of failing fast, so the Node process **hangs**. The internal 1500 ms budget logs a `timeout` event but does **not** exit the process (a handle stays open), so it lingers well past 2 s and the harness hard-kills it, discarding output.
4. Because the process is killed each time, the daemon never comes up and stays down → every subsequent prompt re-hangs. Chronic.

Measured hang (near-zero CPU, pure I/O wait — confirming it's blocked on the absent socket, not compute):

```
recall.js, embeddings enabled, daemon launcher missing:
real 5.12s user 0.18s sys 0.08s (cold)
real 40.5s user 0.19s sys 0.06s (subsequent)
```

## Secondary issues (surfaced after manually restoring embed-daemon.js)

Copying the launcher back in from the plugin bundle (`cp .../bundle/embeddings/embed-daemon.js ~/.hivemind/embed-deps/ && chmod 755`) fixes the hang, but two problems remain that make semantic recall unreliable inside this 2 s-capped synchronous hook:

- **Daemon does not persist between hook invocations.** Runs intermittently take ~20 s — a full cold model reload — meaning the daemon dies between prompts (the 2 s hook-kill likely orphans the freshly spawned child before it detaches).
- **Even warm, recall is 1.6–2.35 s**, i.e. routinely at or over the 2 s cap (Node startup + bundle load ~0.3 s sit on top of the 1.5 s embed+query budget):
```
warm semantic recall: real 1.64s / 1.64s / 2.35s (3rd run trips the 2s cap)
lexical (HIVEMIND_SEMANTIC_SEARCH=false): real 0.16s / 0.16s / 0.34s
```

## Suggested fixes

1. **Self-heal the missing launcher.** When `embeddings.enabled` and `SHARED_DAEMON_PATH` is absent, re-deposit it (or have the plugin `SessionStart`/update step verify+copy it) instead of leaving a permanently broken install. A version bump that keeps `node_modules` but drops `embed-daemon.js` should not be possible.
2. **Fail fast, don't hang.** If the daemon entry file doesn't exist, `trySpawnDaemon`/`waitForSocket` should return immediately and let recall fall back to lexical, rather than looping to the deadline. The recall hook exceeding its own internal budget should hard-exit the process so the harness never has to kill it.
3. **Fix daemon persistence** so it survives across hook invocations (avoid the periodic ~20 s cold reloads).
4. **Reconcile the 2 s cap with semantic latency.** Even a healthy daemon lands ~1.6–2.3 s per call, so semantic recall will intermittently trip the 2 s cap. Consider raising the `recall.js` hook timeout, making it `async`, or gating semantic recall behind a warm-daemon check with lexical fallback otherwise.
5. **Diagnosability:** `hivemind embeddings status` prints `Daemon: (not present)` from an `existsSync(SHARED_DAEMON_PATH)` check on the *script file* (`src/cli/embeddings.ts:305`). That reads as "daemon process not running" when it actually means "launcher file missing." Distinguishing the two would have made this obvious immediately.

## Workaround

For anyone hitting the timeout message: set `HIVEMIND_SEMANTIC_SEARCH=false` (routes recall to the fast lexical path, ~0.2 s, always under the cap). The background summary-embedding path is unaffected (it gates on `embeddings.enabled`, not this env var), so the semantic index still builds once the launcher is restored.

Guia de contribuição

Nenhum guia de contribuição indexado para este repositório

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.