activeloopai / activeloopai/hivemind
embeddings: alternation patterns (a|b|c) skip the embedding daemon and fall back to lexical-only
- Ngôn ngữ chính
- TypeScript
- Star
- 1.6k
- Fork
- 107
- Merge trung bình
- 17 giờ 30 phút
- Pull request đã merge (30 ngày)
- 6
Mô tả
## Severity
Medium-high — limits the embedding feature's main UX (synonym recall is exactly the case embeddings are designed for).
## Where
- `src/hooks/grep-direct.ts:36-50` — `patternIsSemanticFriendly`
- `src/shell/grep-interceptor.ts:43-50` — same function (duplicated)
## Repro
On any workspace with embedding rows, ask Claude Code:
```
HIVEMIND_DEBUG=1 claude -p 'Search hivemind for sessions about silent data loss or concurrent-writer corruption. Use grep -r over the memory path.'
```
CC issues a Bash `grep -r` with an alternation pattern like `silent data loss|data loss|concurrent writer|writer corruption|race condition|lost write`. The debug log (`~/.deeplake/hook-debug.log`) shows **zero** `summary_embedding <#> ARRAY[…]` queries — only lexical `ILIKE` matches and path browses. CC's answer is correct only when the literal keywords match a summary; it misses paraphrases.
## Root cause
`patternIsSemanticFriendly` rejects any pattern with more than 1 regex metacharacter, and `|` counts as a metacharacter:
```ts
const meta = pattern.match(/[|()\[\]{}+?^$\\]/g);
if (!meta) return true;
return meta.length <= 1;
```
When false, the dispatcher skips the embedding daemon call and `searchDeeplakeTables` runs the lexical-only branch.
But `|` is exactly the synonym-list use case where embeddings shine — different surface forms of the same concept. The original intent of the filter ("don't waste an embed call on regex like `[a-z]+\.com`") shouldn't reject synonym alternations.
## Proposed fix
Drop `|` from the metacharacter blacklist + cap the alternative count to guard against pathological inputs:
```ts
function patternIsSemanticFriendly(pattern: string, fixedString: boolean): boolean {
if (!pattern || pattern.length < 2) return false;
if (fixedString) return true;
const meta = pattern.match(/[()\[\]{}+?^$\\]/g); // dropped `|`
if (meta && meta.length > 1) return false;
// Still gate on alternative count to avoid 50-element synonym blasts:
const altCount = pattern.split("|").length;
return altCount <= 8;
}
```
Apply to both `grep-direct.ts` and `grep-interceptor.ts` (or extract into a shared helper while we're at it).
## Acceptance
After fix, the same repro prompt should produce ≥ 1 `summary_embedding <#> ARRAY[…]` query in the debug log, and CC's answer should include sessions matched by paraphrase (not just literal keyword).
## Out of scope
- Recommending Grep tool over Bash grep in the SessionStart prompt (Bash grep already routes through `handleGrepDirect`, which is the embedding path).
- Removing the duplication between the two `patternIsSemanticFriendly` definitions (separate refactor).
Found during the embedding_generation branch validation, see commits 4e956fa + f9f10f3.
Hướng dẫn đóng góp
Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này
Đánh giá
Issue này chưa được đánh giá.