HarperFast / HarperFast/harper

Vector search: return distance-ordered results and expose ANN tuning knobs

Open
#775 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 2h
Merged PRs (30d)
205

Description

## Summary

Vector search via HNSW currently:

1. Returns candidates from the index iterator in **unspecified order**, forcing user code to re-rank by cosine distance to get a meaningful top-N. Every component that uses vector search has to duplicate the same pattern (over-fetch by 5×, compute cosine in JS, sort, slice).
2. Doesn't expose HNSW tuning parameters (`ef_search`, `M`, `efConstruction`) at the schema or per-query layer, so users can't trade recall for latency.
3. Doesn't propagate the distance value back to the caller as part of the row, so user code recomputes it.

## Observed

From the throughput benchmarks on a vanilla harper-pro tenant (193 rows in the index, RTX 4000 Ada host):

| concurrency | rps | p50 | p99 |
|---|---|---|---|
| 1 | 63 | 16ms | 22ms |
| 8 | 247 | 27ms | 65ms |
| 32 | 267 | 104ms | 311ms |

Per-request work for `MatchCelebrity`-style queries is dominated by the **JS cosine re-rank** (over-fetch 50, score 50, sort, slice 10) — not the HNSW traversal itself. On the 193-row table the HNSW search returns in <2ms; the rest is application-level ordering. On a 100k-row table the over-fetch math doesn't scale — we'd need to fetch ~500 candidates to be sure of the top-10, and the re-rank starts to be measurable.

## Proposed Improvements

### 1. Return distance with each row, in distance order

```js
const iter = tables.X.search({
conditions: { attribute: 'embedding', target: queryVec },
limit: 10,
})
for await (const r of iter) {
// r.embedding is the vector (current behaviour)
// r._distance is the cosine distance from queryVec (NEW)
// iteration order is ascending distance (NEW)
}
```

Saves every consumer from over-fetching + JS re-sorting. Brings parity with how every other HNSW library (FAISS, hnswlib, Milvus, pgvector) returns results.

### 2. Expose `ef_search` per query

```js
tables.X.search({
conditions: { attribute: 'embedding', target: queryVec },
limit: 10,
hnsw: { efSearch: 200 } // larger = higher recall, more candidates examined
})
```

`ef_search` is the standard HNSW recall/latency knob. The library already accepts it; we just don't surface it.

### 3. Expose `M` and `efConstruction` on the schema

```graphql
embedding: [Float] @indexed(type: "HNSW", distance: "cosine", m: 32, efConstruction: 200)
```

Today `M` (graph connectivity) and `efConstruction` (build-time accuracy) appear to be defaulted internally. Workloads with very different recall requirements (e.g. memory-tight edge tenants vs. high-recall search-as-a-service tenants) can't tune them.

### 4. Add quantization options

`int8` PQ / SQ quantization is table-stakes for any vector store handling ≥1M rows. Cuts memory ~4× with ~1–2% recall loss. Would let a single Harper tenant comfortably hold 10M+ vectors per worker before paging.

### 5. Document `distance` semantics clearly

The current API uses `comparator: 'lt', value: 2, target: vec` to mean "cosine distance < 2" — which is just "everything", since cosine distance is bounded [0,2]. The `value` is the filter cutoff. This isn't documented anywhere I could find, and users hit it via copy-paste rather than understanding. A worked example with a real distance threshold would help.

## Priority/Impact

Medium-to-high. Once Harper starts pitching "vector search inside your database" as a first-class story (and it should — the benchmarks are good), the per-query ergonomics need to match what users expect from pgvector / Pinecone / Milvus. (1) and (2) are the table-stakes; (3)/(4)/(5) can follow.

## Related

- harper-celebrity-match `resources/MatchCelebrity.js` — duplicates over-fetch-and-re-rank
- harper-roadmap `resources/Search.js` — duplicates the same pattern
- agent-example-harper — same pattern

Every demo we've built has shipped the same boilerplate around `tables.X.search({ conditions: { attribute, comparator: 'lt', value: 2, target } })`. That's a sign the API needs work.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the JavaScript search API described in the issue and compare its behavior with resources/MatchCelebrity.js, resources/Search.js, and agent-example-harper. Clarify which proposal is in scope first, then identify the HNSW schema and query entry points and existing tests; done should include an agreed API, ordered results with distance, tuning behavior, and documented distance semantics.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, nodejs
Domain
backend-api-design, databases, performance, search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.