HarperFast / HarperFast/harper

Resource layer: fail-closed optimistic precondition for writes (atomic create-if-absent + version CAS)

Open
#2,149 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 2h
Merged PRs (30d)
205

Description

## Summary

Harper's resource layer has no fail-closed **optimistic precondition** on writes — no "create only if absent" that actually rejects a concurrent peer, and no "put/patch only if version == expected" compare-and-set. This means two idioms app developers reach for do **not** provide the mutual exclusion they appear to:

- **`create()` + catch-409** — the duplicate check is a `getSync` on *committed* state (`resources/Table.ts` ~2152), the write stages `deferSave:true` and drains at commit, and on an `ERR_BUSY`/`RETRY_NOW` conflict the staged write is re-committed **without re-running the handler** (`resources/DatabaseTransaction.ts` ~577-616). The create commit closure is a plain put with no existence precondition, so two concurrent same-PK creates both land — **no 409 on the loser**.
- **get → compare version → patch** — there is no app-supplied `ifVersion`/`expectedVersion` on the put/patch API; `ifVersion` is only used internally on the out-of-transaction cache-resolve path and is recomputed from the reloaded entry on retry. Two readers both see `version: 0`, both patch, and the loser is re-based rather than rejected.

Both were confirmed by source trace against Harper 5.2.0 and reproduced against a real RocksDB instance (two concurrent `create()`s → last-write-wins, no 409; a compare-and-set REVOKE lost to a concurrent CONFIRM against the same version).

## Why this is an enhancement, not a fire

An app that needs single-writer-per-key correctness can't express it declaratively today; it must work around the gap (route resolution through a deterministic-PK arbiter row + reap duplicates, or serialize writes app-side). That's viable but repeated boilerplate, and it's easy to *assume* `create()`+catch-409 is atomic when it isn't. A first-class primitive would remove a sharp edge.

## Proposed

A fail-closed optimistic precondition surfaced on the resource write API — `create()` rejecting when the key is present at commit, and `put`/`patch` accepting an `expectedVersion` that throws a conflict when the committed version differs. The plumbing largely exists (the commit closure already receives the reloaded entry + retry count, and `putOptions.ifVersion` already reaches the store); the missing piece is honoring an app-supplied expected value on the retry round instead of re-basing.

## Cross-node note

Even a perfect single-node precondition does not stop two *different* nodes creating the same PK — replication resolves same-PK conflicts by timestamp last-write-wins. True cross-node uniqueness needs single-owner routing or distributed coordination; that's out of scope for this primitive but worth stating so it isn't assumed.

## Context

Surfaced during a Harper central-manager OAuth authorization feature whose design assumed `create()`+catch-409 and version-CAS were atomic. The feature was made correct without this primitive (deterministic-PK arbiter + orphan reaping); this issue tracks the general capability so future app code doesn't have to.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Contributor guide

Open the contributing guide

Research direction

Start by tracing the resource write path in resources/Table.ts around the getSync and create logic, then follow retry handling in resources/DatabaseTransaction.ts around lines 577-616. Inspect how putOptions.ifVersion reaches the store and reproduce the concurrent create and version-CAS scenarios against RocksDB. Done means committed create-if-absent and expectedVersion writes reject conflicts rather than being rebased, while the cross-node limitation remains explicit.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, nodejs
Domain
api, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.