HarperFast / HarperFast/harper
Vector search: return distance-ordered results and expose ANN tuning knobs
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 205
Description
## Summary
Vector search via HNSW currently:
1. Returns candidates from the index iterator in **unspecified order**, forcing user code to re-rank by cosine distance to get a meaningful top-N. Every component that uses vector search has to duplicate the same pattern (over-fetch by 5×, compute cosine in JS, sort, slice).
2. Doesn't expose HNSW tuning parameters (`ef_search`, `M`, `efConstruction`) at the schema or per-query layer, so users can't trade recall for latency.
3. Doesn't propagate the distance value back to the caller as part of the row, so user code recomputes it.
## Observed
From the throughput benchmarks on a vanilla harper-pro tenant (193 rows in the index, RTX 4000 Ada host):
| concurrency | rps | p50 | p99 |
|---|---|---|---|
| 1 | 63 | 16ms | 22ms |
| 8 | 247 | 27ms | 65ms |
| 32 | 267 | 104ms | 311ms |
Per-request work for `MatchCelebrity`-style queries is dominated by the **JS cosine re-rank** (over-fetch 50, score 50, sort, slice 10) — not the HNSW traversal itself. On the 193-row table the HNSW search returns in <2ms; the rest is application-level ordering. On a 100k-row table the over-fetch math doesn't scale — we'd need to fetch ~500 candidates to be sure of the top-10, and the re-rank starts to be measurable.
## Proposed Improvements
### 1. Return distance with each row, in distance order
```js
const iter = tables.X.search({
conditions: { attribute: 'embedding', target: queryVec },
limit: 10,
})
for await (const r of iter) {
// r.embedding is the vector (current behaviour)
// r._distance is the cosine distance from queryVec (NEW)
// iteration order is ascending distance (NEW)
}
```
Saves every consumer from over-fetching + JS re-sorting. Brings parity with how every other HNSW library (FAISS, hnswlib, Milvus, pgvector) returns results.
### 2. Expose `ef_search` per query
```js
tables.X.search({
conditions: { attribute: 'embedding', target: queryVec },
limit: 10,
hnsw: { efSearch: 200 } // larger = higher recall, more candidates examined
})
```
`ef_search` is the standard HNSW recall/latency knob. The library already accepts it; we just don't surface it.
### 3. Expose `M` and `efConstruction` on the schema
```graphql
embedding: [Float] @indexed(type: "HNSW", distance: "cosine", m: 32, efConstruction: 200)
```
Today `M` (graph connectivity) and `efConstruction` (build-time accuracy) appear to be defaulted internally. Workloads with very different recall requirements (e.g. memory-tight edge tenants vs. high-recall search-as-a-service tenants) can't tune them.
### 4. Add quantization options
`int8` PQ / SQ quantization is table-stakes for any vector store handling ≥1M rows. Cuts memory ~4× with ~1–2% recall loss. Would let a single Harper tenant comfortably hold 10M+ vectors per worker before paging.
### 5. Document `distance` semantics clearly
The current API uses `comparator: 'lt', value: 2, target: vec` to mean "cosine distance < 2" — which is just "everything", since cosine distance is bounded [0,2]. The `value` is the filter cutoff. This isn't documented anywhere I could find, and users hit it via copy-paste rather than understanding. A worked example with a real distance threshold would help.
## Priority/Impact
Medium-to-high. Once Harper starts pitching "vector search inside your database" as a first-class story (and it should — the benchmarks are good), the per-query ergonomics need to match what users expect from pgvector / Pinecone / Milvus. (1) and (2) are the table-stakes; (3)/(4)/(5) can follow.
## Related
- harper-celebrity-match `resources/MatchCelebrity.js` — duplicates over-fetch-and-re-rank
- harper-roadmap `resources/Search.js` — duplicates the same pattern
- agent-example-harper — same pattern
Every demo we've built has shipped the same boilerplate around `tables.X.search({ conditions: { attribute, comparator: 'lt', value: 2, target } })`. That's a sign the API needs work.
Contributor guide
Research direction
Start by tracing the JavaScript search API described in the issue and compare its behavior with resources/MatchCelebrity.js, resources/Search.js, and agent-example-harper. Clarify which proposal is in scope first, then identify the HNSW schema and query entry points and existing tests; done should include an agreed API, ordered results with distance, tuning behavior, and documented distance semantics.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, nodejs
- Domain
- backend-api-design, databases, performance, search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100