HarperFast / HarperFast/harper
Support server-side text auto-embedding at query time for vector/k-NN search
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 200
Description
## Summary
Today, vector/k-NN search (via the Query `sort: { attribute, target, distance }` clause, and the proposed `search_by_conditions` extension in #1826) requires the caller to supply a pre-computed numeric vector as `target`. There is no server-side path that accepts a plain text string and auto-embeds it at query time using the column's configured `@embed` model — even though the write side (`@embed`) already does exactly this kind of auto-embedding on `put`/update.
This is a real ergonomics gap: a caller doing semantic search over an `@embed` column has to independently know/call the same embedding model and pipeline Harper already used to build the index, rather than just passing search text.
## Current behavior (confirmed)
- `resources/indexes/HierarchicalNavigableSmallWorld.ts:974-975` hard-rejects non-array targets:
```ts
if (!target) throw new ClientError('A target vector must be provided for an HNSW query');
if (!Array.isArray(target)) throw new ClientError('The target vector must be an array');
```
- `resources/Table.ts` sort handling (~line 2787-2825) passes `sort.target` through untouched via a shallow spread (`orderAlignedCondition = { ...sort, comparator: 'sort' }`) — no coercion, no embed call.
- `Models.embed` (`resources/models/Models.ts:93`) is only ever invoked from the write-time hook (`resources/models/embedHook.ts:43,52-66`, wired into `Table.ts`'s pre-commit `put`/update path via `buildEmbedBefore`). It is never called from the read/search path (`search.ts`, `HierarchicalNavigableSmallWorld.ts`, `Table.ts` search).
- There's a dedicated integration test proving this negative today: `integrationTests/qa-scratch/qa457-embed-read-leg.test.ts` sends `sort: { attribute: 'embedding', target: 'a red running shoe', distance: 'cosine' }` and asserts a clean synchronous 4xx with **zero calls reaching the embedding backend** (`mock.log.length === 0`).
- Notably, `resources/models/types.ts` already defines an `inputType: 'query'` option on the embedding-model config — clearly intended for exactly this scenario (asymmetric embedding models that encode query text differently from document text) — but it is **never actually passed or wired anywhere** in the codebase. It's dormant scaffolding for this feature. `qa457-embed-read-leg.test.ts` (lines 27-32) calls this out explicitly.
## Proposal
Add an opt-in server-side text-embedding path for vector search:
- If `sort.target` (or the k-NN condition's `target` once #1826 lands) is a `string` instead of `number[]`, and the target attribute has a `@embed`-configured model, call `Models.embed(text, { model, inputType: 'query' })` before constructing the HNSW search condition — reusing the same `models.embed` machinery `embedHook.ts` already uses on write.
- Wire up the dormant `inputType: 'query'` config knob so asymmetric embedding models (e.g. models with distinct query/document encoders) embed search text correctly, distinct from how document text was embedded at write time.
- Needs design decisions: is the embed call awaited synchronously in the search hot path (latency/backend-failure handling — note write-side already has retry/backoff patterns in `embedHook.ts` worth reusing), does this apply to REST/GraphQL only or also `search_by_conditions` (depends on #1826), and how errors/backend timeouts surface to the caller (`qa457` test's persistent-failure and hang-mode cases model the failure surface to preserve).
## Related
- #1826 — search_by_conditions can't do vector/k-NN search on @embed / HNSW-indexed columns (this issue is a superset concern: even once that's fixed, callers still can't pass raw text)
Contributor guide
Research direction
Start with resources/indexes/HierarchicalNavigableSmallWorld.ts, resources/Table.ts, resources/models/Models.ts, and resources/models/embedHook.ts; run integrationTests/qa-scratch/qa457-embed-read-leg.test.ts to understand the current rejection and failure cases. First resolve the synchronous error-handling and search_by_conditions scope decisions described in the issue. Done means string query targets use the configured query embedding path, asymmetric models are covered, and qa457's failure behavior remains tested.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- database, search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 42/100