HarperFast / HarperFast/harper
Blob content resolution as an internal Resource (owned-relationship model, pluggable sources)
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 205
Description
Builds on #2146 (location-independent blob identity). That issue proposes the identity primitive — `(node_id, fileId)`, node-namespaced storage, content that moves independently of records. This issue proposes the shape of the layer that resolves and serves that content: **an internal, custom-optimized Resource**.
## Data model: a blob attribute is an owned relationship
A relationship field is a reference by stable id to an entity owned by another resource, resolved lazily on access, where the target resource controls its own storage, caching, and sourcing. With stable identity, a blob attribute is exactly that — an **owned relationship** into an internal blob resource — currently implemented as a hardwired special case (the msgpackr blob extension + private file-id bookkeeping) instead of through the reference model Harper already has.
The wire consequence is the notable one: relationship fields already replicate as bare ids — the target never rides along; it arrives through its own resource's replication. Blob attributes are today the *only* reference type whose target is force-inlined into the record stream. "Send refs, move content independently" (#2146 phase 3) is therefore not a new wire mode — it is making blobs consistent with how every other reference in the system already replicates.
"Owned" (composition) rather than an ordinary association: the blob is written inside the owning record's commit (pre-commit durability gate) and deleted by record supersession — 1:1, no refcounting. If cross-record blob sharing ever lands, the relationship graduates to a normal association and refcounting GC becomes due at that moment, not before.
## The Resource
A system-scoped resource (working name `hdb_blobs`), keyed by `(node_id, fileId)`, backed by the node-namespaced blob file tree — not by a database.
- **`get()` returns the Blob** — the existing `FileBackedBlob` streaming machinery is the value type, so the HTTP layer streams it with correct Content-Length for free. Local hit: open the file. Miss: resolve through the configured source, single-flight per id, materializing to the final path so concurrent readers tail the file as it lands (the read-while-writing machinery in `resources/blob.ts` already provides this).
- **Peer fetch stops being new surface.** `GET /hdb_blobs/{node_id}/{fileId}` between nodes is just a Resource read — routing, cluster-cert auth, and body streaming already exist. Range support (for resumable large fetches) is cheap to add since `blob.slice()` already implements offset reads.
- **Repair collapses into reads.** The repair sweep (harper-pro#385/#388) becomes "iterate refs missing files, call `hdb_blobs.get()`" — no dedicated repair transport, no borrowed replication connections (harper-pro#684's structural problem disappears rather than being fixed).
- **Sources are pluggable — this is where placement policy lives** (#2146's identity/placement separation, expressed in class design):
- *Cluster peers* (default): latency-ordered holder selection with failover — closest first, then the minting node, then any known node. Reuses the selection model record `sourcedFrom` resolution already uses.
- *Object store* (later): `(node_id, fileId)` maps 1:1 onto a bucket keyspace; cold tiering arrives through the same source interface, not a redesign.
- *Origin URL* (later): CDN-style fill for cache workloads.
Because content is immutable, this resource is much simpler than a caching table: no invalidation, no version conflicts on load, no MVCC — resolution is "have it or fetch it."
## Resolution lifecycle
Per-blob state at a replica:
```mermaid
stateDiagram-v2
[*] --> Absent: record applied with ref (A, f1)
Absent --> Fetching: push arrives / prefetch / read miss / sweep
Fetching --> Present: bytes complete (atomic rename)
Fetching --> Absent: attempt failed — partial kept for Range resume, retry later
Absent --> Dropped: newer record version drops ref (record is the tombstone)
Fetching --> Dropped: ref superseded — cancel fetch
Present --> [*]: record superseded → normal GC
note right of Fetching
single-flight per (node_id, fileId)
readers tail the materializing file
peer order: closest → origin → any holder
404 from all peers + ref still current → repair alert
end note
note right of Present
read = local serve
end note
```
Resumability and resilience come from three properties: **immutability** (`(node_id, fileId)` content never changes, so re-fetch and `Range` resume are always safe, from any holder), **single-flight + tailing** (concurrent triggers — push, prefetch, read miss, sweep — coalesce into one transfer; concurrent readers stream from the file as it lands), and **record-version-slaved lifetime** (no fetch outlives the ref that justified it; supersession cancels it, so absence classification needs no tombstone infrastructure — the record is the tombstone).
## Transport for the peer source
Pooled mTLS HTTP between nodes, **not** the replication WebSocket. Investigation of the current code found the replication socket structurally unsuited to request/response blob traffic: inbound frames are processed strictly serially per connection, receiver backpressure pauses the whole socket, blob sends share a small per-connection concurrency cap with record replication, `GET_RECORD` has no request deadline, and there is no blob-only frame at all — fetching a blob today drags its entire record, and a serve-path miss has no fetch path whatsoever. A dedicated framed channel would pay identical per-byte costs (TLS dominates; Node cannot sendfile through TLS) while rebuilding multiplexing, per-stream flow control, and fairness from scratch — the machinery that produced the existing pause/wedge bug family.
Cost notes: blobs under the 8KB inline threshold never hit this path (they ride in the record), so per-request HTTP overhead is bounded at ≤~5% and falls toward zero for large blobs; mTLS cost is handshake-only and amortized by a bounded keep-alive pool per peer (TLS session tickets make reconnects cheap). If sustained high-rate transfer ever shows per-request overhead in profiles, a batch form (one request streaming N blobs, length-prefixed) is the escape hatch — the batch shape (Resource query vs. small custom handler) is an open detail.
## What stays outside the Resource
- **The hot local path.** Same-node reads keep going record → `storageInfo` → file directly; the Resource is the network facade and the miss-resolution point, not a per-read indirection. Layering: core `blob.ts` cannot depend upward on the Resource layer, so fetch-on-miss goes through a registered resolver hook (the `currentBlobCallback` registration pattern already models this) that the Resource installs at startup.
- **Minting and GC.** Blob creation stays in `saveBlob` inside record encoding with the pre-commit gate; deletion stays record-supersession-driven. The Resource writes only as the materialization step of a fetch (temp write, atomic rename).
- **Public exposure.** System-scoped, cluster-auth only. A resource that serves arbitrary content by id must not appear in the default public REST surface.
## Composition
Record miss → table `sourcedFrom` (unchanged) → record lands carrying `(node_id, fileId)` → blob miss → `hdb_blobs` source resolution. The same resolve-through-source pattern at two granularities, each with its right transport. A record's metadata and its blob can no longer desync under concurrent resolution (harper-pro#645): the blob key is taken from the resolved record, so the two cannot disagree.
Related: #2146 (identity primitive this builds on), #202, #1645, #2134, #141; harper-pro#385, #388, #684, #645, #208, #158, #195.
---
Priority: low — design/direction proposal, companion to #2146. _Drafted with AI assistance (Claude Code)._
Contributor guide
Research direction
Start by reading resources/blob.ts and core blob.ts, especially the existing read-while-writing machinery and currentBlobCallback pattern, then inspect saveBlob and sourcedFrom resolution. Compare the proposed hdb_blobs Resource, peer HTTP source, lifecycle, and cancellation requirements with the related issues and existing transport code. Done means the design's open details are resolved into an agreed implementation plan.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js
- Domain
- backend, databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100