Conversational retrieval primitives: seed-anchored search, multi-vector targets, MMR diversification, per-query ef

# Conversational retrieval primitives: seed-anchored search, multi-vector targets, MMR diversification, per-query ef

## Problem

RAG retrieval quality degrades sharply on conversational follow-ups: "what about its performance?" embeds to nothing useful on its own, and each turn re-retrieves the same top chunks it already served. The fixes are well known — carry conversation context into the query, stay in the neighborhood the conversation is exploring, and diversify against already-served content — but they need server-side primitives the HNSW search doesn't expose yet. The session/orchestration state lives in the caller (an agent loop, e.g. #612's `toolMode: 'auto'`, or a `ConversationResource` per #511); what core needs to provide is a search API those callers can drive.

Four primitives, smallest first. They are independent — each lands on its own — but they're specified together because they share the query surface and the conversational use case.

## 1. Per-query `ef`

`HierarchicalNavigableSmallWorld.search()` already accepts a per-query `ef` (it wins over `efConstructionSearch` and auto-scaling — see the ef-resolution block at `HierarchicalNavigableSmallWorld.ts:696–702`), but there is no documented query surface to set it. Expose it on the sort object:

```js
{ sort: { attribute: 'embedding', target: v, ef: 200 } }
```

Plumbing only: the sort object already flows into the index condition (`Table.ts:2357` builds the pseudo-condition `{ ...sort, comparator: 'sort' }`). Validate it's a positive integer with a sane cap (e.g. 10 × default, configurable) so a public query string can't force pathological traversals.

## 2. Seed-anchored search (warm-start from prior results)

The HNSW entry point is just a node, and `searchLayer()` can start from any node. Let a query supply seed records — typically the previous turn's retrieval hits — and initialize the layer-0 beam from them in addition to the normal entry-point descent:

```js
Chunks.search({
sort: {
attribute: 'embedding',
target: currentQueryVector,
seeds: ['chunk-41', 'chunk-87'], // primary keys from the previous turn
},
limit: 10,
});
```

Semantics: the search runs the standard upper-layer descent to find the global entry candidate, then at layer 0 the candidate heap is initialized with `seeds ∪ {descendedEntryPoint}` (each seed's distance to the target computed on entry; seeds that don't resolve are ignored silently — records may have been deleted). Results are whatever the beam converges to; seeds confer no score bonus, they only shape where the search *starts*, which both speeds convergence and biases recall toward the conversation's active neighborhood.

Implementation:

- Seed primary keys resolve to node IDs via the existing `[Symbol.for('key'), primaryKey] → nodeId` mapping (`safeKey` lookup, see `index()` around line 196).
- `searchLayer()` (line 553) currently takes a single `entryPointId`/`entryPoint`; generalize to accept an initial candidate list (the visited set seeded accordingly). The single-entry case is the degenerate call, so this is a small refactor, not a rewrite.
- Keeping the global descent alongside the seeds matters: seeds alone can trap the beam in a stale neighborhood when the topic shifts. The union is self-correcting — if the global descent lands somewhere better, the beam follows it.

This is cheap, very HNSW-native, and I'm not aware of mainstream vector stores exposing it; it falls out of Harper owning the graph structure directly.

## 3. Multi-vector targets

Support searching against several vectors with a combined distance — e.g. the current query plus a decayed conversation centroid, or a query plus a "must also relate to X" anchor:

```js
{
sort: {
attribute: 'embedding',
target: [queryVector, contextCentroid], // array of vectors
weights: [0.7, 0.3], // default: uniform
combine: 'weightedSum', // or 'min'
},
}
```

- `combine: 'weightedSum'` — for normalized cosine this is nearly equivalent to pre-averaging the vectors client-side (which remains the recommended cheap path and should be documented as such); server-side support exists mainly for symmetry and for callers that can't pre-process.
- `combine: 'min'` — "near *any* of these targets." This one cannot be emulated client-side with a single query, and it's the useful multi-anchor semantic.
- Implementation: build a closure over the per-target distance functions and combine; pass it as the `distanceFunction` parameter `searchLayer()` already accepts (line 553 takes `distanceFunction = this.distance`). The int8 asymmetric-distance fast paths (lines 566–602) are per-target; the combiner wraps them. The exact-rerank path (`exactDistance`, line ~732, used by the int8 rerank in `search.ts`) needs the same combination logic so reranked ordering matches traversal semantics.
- Input validation: all targets must match the indexed dimension; cap target count (e.g. 8).

## 4. MMR diversification + exclusion set

Re-serving the same chunks every turn wastes context-window budget. Add a diversification rerank stage:

```js
{
sort: { attribute: 'embedding', target: v },
diversify: {
lambda: 0.5, // 1.0 = pure relevance, 0.0 = pure diversity
against: servedIds, // primary keys to diversify away from / exclude
exclude: true, // if true, `against` ids are dropped outright (default true)
},
limit: 10,
}
```

- Standard greedy MMR over the post-rerank candidate pool: repeatedly pick `argmax(λ · relevance − (1−λ) · maxSimilarityToSelected)`, where the selected set is initialized from `against` when `exclude: false` (penalize-but-allow) or `against` is removed from the pool when `exclude: true`.
- Implementation site: after the existing exact-distance rerank block in `search.ts` (the int8 rerank around lines 405–423) and before limit/offset slicing (`Table.ts:2434`). Requires candidate vectors — available on the loaded records (the full-precision embedding attribute), which the rerank path already loads.
- Over-fetch: MMR needs a pool larger than `limit`; fetch `max(ef, 4 × limit)` candidates into the pool. Document the interaction with `ef`.
- Pairwise similarity cost is O(pool × limit) distance computations — fine at pool ≤ a few hundred; cap the pool.

## The conversational pattern (documentation deliverable)

With 1–4 in place, a caller-side conversation loop looks like:

```js
// per turn:
const queryVec = await embed(userMessage);
centroid = decay(centroid, 0.5).add(queryVec).normalize(); // caller-maintained
const results = await Chunks.search({
sort: { attribute: 'embedding', target: [queryVec, centroid], weights: [0.7, 0.3], seeds: lastHits },
diversify: { against: servedIds },
limit: 8,
}, context);
lastHits = results.map(r => r.id);
servedIds.push(...lastHits);
```

This belongs in the docs as a worked example, and it is the natural retrieval tool for the in-process agent loop (#612): a Resource-backed `search_chunks` tool whose handler threads `seeds`/`against` from conversation state (e.g. a `ConversationResource`, #511). Core ships the primitives; the loop owns the state.

## Acceptance

- [ ] `ef` settable per query via the sort object (JS + REST), validated and capped.
- [ ] `seeds` initializes the layer-0 beam alongside the global descent; unresolvable seeds ignored; topic-shift test confirms global descent prevents neighborhood lock-in.
- [ ] Multi-vector `target` with `weights` and `combine: 'weightedSum' | 'min'`; exact rerank uses the same combined distance; dimension/count validation.
- [ ] `diversify` MMR rerank with `lambda`, `against`, `exclude`; pool over-fetch documented and capped.
- [ ] All four compose with predicate-filtered traversal (series part 1) and relationship expansion (part 2) where applicable — at minimum, no crashes and sane semantics when combined; document combinations that are intentionally unsupported.
- [ ] Tests in `unitTests/resources/vectorIndex.test.js` style for each primitive plus the composed conversational pattern.
- [ ] Benchmark: multi-turn retrieval scenario (scripted conversation over a fixture corpus) measuring gold-context hit-rate per turn vs. plain per-turn ANN; seed-anchored + centroid + MMR should beat the baseline on turns 2+.
- [ ] Docs: option reference plus the worked conversational-RAG pattern above, cross-referenced from the model/agent docs.

## Out of scope

- Server-side session state (conversation centroids, served-id tracking). Deliberately caller-owned — it composes with #511/#612 rather than duplicating them.
- Query rewriting / condensation via an LLM before embedding (an orchestration concern; `scope.models` callers can do it).
- Time-decay or recency weighting baked into the index.

## Related

- #612 — agent-loop orchestration (`toolMode: 'auto'`): the primary downstream consumer; a retrieval tool in that loop drives `seeds`/`against` from conversation state.
- #511 — `ConversationResource`: natural home for the caller-side state in the worked pattern.

## Series

Part 3 of the hybrid graph/semantic retrieval sequence: #1241 (predicate-aware traversal) → #1242 (relationship expansion) → this → #1244 (semantic hierarchy research). Independent of parts 1–2 (no hard dependency), but composes with both.

---

🤖 Generated with [Claude Code](https://claude.com/claude-code)