HarperFast / HarperFast/harper
[Models] Derived caching table for @embed (low-latency writes + model-change backfill)
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 200
Description
## Scope
Follow-up to #632 (Phase 5 of #510). #632 shipped `@embed` with **sync-by-default**: the embedder runs during the transaction's `before` phase, so commit blocks on it. This is the right default for the common case (per @kriszyp on #632: "overwhelmingly the mode that should and would be used").
This issue tracks the **derived caching table** pattern for the cases where write latency matters: origin-table writes commit immediately, and a separate derived table holds the vector and follows sync mode against the origin.
> I am more inclined to think the right way to minimize write latency is to move the vector embedding into a derived caching table (that follows the sync mode, but the "origin" table doesn't need to wait for it).
> — @kriszyp on #632
## Why this is meaningful
Sync mode adds an HTTP round-trip (to the embedding backend) to every originating write that touches the source field. For latency-sensitive write paths (high-throughput ingest, user-facing POSTs), that's not acceptable even with a fast local backend. The derived-cache-table pattern keeps origin-write latency at LMDB-speed while still producing the vector in the background — without ad-hoc job infrastructure on top of `server/jobs/`.
## Four things, one primitive
The same iterate-records-and-rewrite primitive (a derived table that embeds by observing origin content changes) lands **four** capabilities #632 deferred:
1. **Low-latency writes** — origin commits without the embedder; derived table embeds and stores asynchronously.
2. **Re-embed-on-model-change backfill** — when `@embed(model: ...)` changes between deploys, existing rows need to be re-embedded through the new model. #632 today only re-INDEXES existing vectors via the schema version-change pathway; it does NOT re-embed. The derived table is the natural place to scan-and-rewrite.
3. **Proactive source-subscribe embedding** — closes a known gap in #632. A caching table sourced from an external source that pushes proactive `put`/`patch` events via `subscribe()` writes them with `isNotification: true`, which `buildEmbedBefore` skips (correctly, for cluster-replication receivers — but the source-push case carries content with no precomputed vector). The skip only bites when the table does not override `get()` (`shouldRevalidateEvents === false`); with a custom `get()`, subscribe events invalidate and the next read re-embeds via `getFromSource`. A derived table that embeds from observed origin content closes this regardless of how the origin was updated (user write, `getFromSource`, or subscribe push).
4. **Write-time-hook timing (default sync mode)** — closes a known limitation in #632. The hook runs in `_writeUpdate` against the in-flight write payload, *before* table validation, so: (a) a write that later fails validation still calls the embedding backend (wasted cost on the reject path), and (b) a tracked-instance mutation (`update(id, {}); row.source = ...; save()`) that sets the source via accessors after `update()` won't re-embed — the source field isn't in that payload. The clean fix — embed at validate-time / the resource layer, against the final committed record — was prototyped and reverted on #632 as a Harper-foreign pattern. Embedding from the *committed, validated* record (which the derived-table / resource-layer approach above does) closes both symptoms: invalid writes never commit (so never observed → no backend call), and the observed record is the final post-merge content.
**Cache-fill failure semantics (note):** if the embedder fails during a `getFromSource` cache-fill, the background cache write aborts *after* the GET has already resolved (the caller already received the source data) — so the row is left uncached and re-embeds on the next read, rather than caching a vectorless row. The derived-table / resource-layer approach should preserve this "don't persist a broken vector" behavior, and is the natural place to add retry/backfill if desired.
## Implementation sketch (to be refined in PR review)
- Component-author flag on `@embed` (or table-level config) opt-in: `@embed(source: ..., model: ..., mode: "derived")` or equivalent. Default stays sync.
- For derived-mode `@embed` attributes:
- Origin write at the put/patch site does NOT chain the embedder into the `before` phase.
- Origin commit triggers an event observed by a paired derived table (Harper resource pattern).
- Derived table's write path runs the embedder in its own `before` phase (i.e. derived-table writes ARE sync against the origin, just deferred).
- Lookup of the vector for query-time follows the derived-table key (origin's PK).
- Model-change backfill: when `attribute.embed.model` differs from `attributeDescriptor.embed.model` at schema load, iterate the derived table and re-embed each row. Sync mode at table load (matches #632's stated default) for small tables; can be lifted to background scan for large.
## Stacks on
- #632 (Phase 5), PR #747 — `@embed` directive, default embedder, write-time hook, auto-HNSW, replication-receiver predicate, model-tracking version. Lands first; this issue stacks on it once it merges.
## Out of scope
- Cross-model migration tooling (`harper kb reembed --from X --to Y`) — separate concern.
- Multimodal embeddings (image, audio).
## Tracking
Part of #510. Follow-up to #632.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Contributor guide
Assessment
This issue has not been assessed yet.