HarperFast / HarperFast/rocksdb-js

Dual-clock support in rocksdb-js: expose the distinct-version word, harden setTimestamp, seed the clock floor at open (no commit-time stamping)

Open
#811 1 comment 0 reactions 1 assignee Claimed by @kriszyp View on GitHub
Dominant language
C++
Stars
21
Forks
2
Avg merge
2d 9h
Merged PRs (30d)
36

Description

## Direction (rewritten 2026-09-02)

Supersedes the commit-time stamping design; [feat: commit-time local mutation stamping behind a dormant per-column-family gate (#813)](https://github.com/HarperFast/rocksdb-js/pull/813) is closed unmerged, branch kept for salvage.

Harper's write identity stays the transaction timestamp rocksdb-js assigns at construction (`TransactionHandle::startTimestamp`, strictly increasing per process). It is the record's first word (the verification table keys freshness on it) and the transaction-log batch key, and the two must be equal so a record can find its own log entry: harper's `auditStore.get(firstWord, tableId, id, nodeId)` seeks the per-origin log at exactly that key. Identity is the pair (origin nodeId, first word): uniqueness only has to hold within one origin's log, which that origin's monotonic clock provides. A replication receiver and crash replay adopt the origin's first word through `setTimestamp`; nothing else may.

A source- or origin-supplied record version (caching `lastModified`, a replicated origin version) is **not** the first word. It rides in a distinct second word written by harper's encoder under `HAS_DISTINCT_VERSION_FLAG`, present only when it differs from the first word ([harper#2412](https://github.com/HarperFast/harper/issues/2412) stage 2). rocksdb-js stamps nothing at commit.

**Why not commit-time stamping.** Keep-if-greater re-stamps every transaction outlived by a concurrent commit (#813 design §3.2; B10 measured an 87.5% re-stamp rate by construction; B4 ~1.45× and 2× batch bytes for a 10k-put batch). Under load P(keep) ≈ e^(−throughput × latency), so large long-running transactions always pay the rebuild. What it buys is claim-order-monotonic log keys, which no consumer needs: all of them use equality or log position, and `findPositionByTimestamp` already handles non-monotonic keys as a running-maxima seek.

## Deliverables

1. **Read surface.** `getEntry`/`getEntrySync` return both words: `localTime` (first word) and `version` (the second word when `HAS_DISTINCT_VERSION_FLAG` = `0x20000` is set in the metadata, otherwise equal to the first word). Export the flag. The verification table keeps keying on the first word; no VT change.
2. **`setTimestamp` hardening** (the dormant-visible fixes from #813, carried over with their tests): reject non-finite values and values ≥ 8.64e15; reject calls on a non-`Pending` transaction, which also removes the JS-thread vs commit-lane race on `startTimestamp`.
3. **Clock floor at open.** Seed `getMonotonicTimestamp`'s floor from the largest durable log key across the database's transaction logs, so a wall-clock rollback across restart cannot reissue a first word already present in this node's log. No metadata CF, no backup artifact.
4. **Docs.** `startTimestamp` is the write identity within this node's log. `setTimestamp` exists to adopt an origin's timestamp on replicated apply and replay only. Log keys are not monotonic under concurrency (already true today) and `findPositionByTimestamp` is a running-maxima seek.

## Non-goals

Commit-time claim or re-stamp, a per-column-family enable gate, the `__rocksdbjs.meta` CF, a per-hop cursor flip, and retiring `VERSION_NOT_UNIQUE_FLAG` (#766 stays): a same-key write from a different origin at an equal first word still rewrites at that word, and harper keeps setting the flag for it.

## Companions

[harper#2412](https://github.com/HarperFast/harper/issues/2412) (format + call-site audit), [harper-pro#790](https://github.com/HarperFast/harper-pro/issues/790) (receiver adopts first word and version separately), [harper#2411](https://github.com/HarperFast/harper/issues/2411) (replay).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.