HarperFast / HarperFast/harper

[Design] No safe non-commutative concurrency primitive (compare-and-set / claim-once / clamped decrement) on a single node

Open
#1,331 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 6h
Merged PRs (30d)
200

Description

## Summary
A consolidation of the campaign's concurrency findings into one design/capability observation: **Harper v5 single-node has no safe non-commutative concurrent-write primitive.** The only primitive whose committed result is exact under single-key contention is the commutative atomic delta (`addTo`/`subtractFrom`). Every compare-and-set-shaped idiom is unsafe under concurrency (RocksDB rare/narrow, LMDB on every round), because each takes a snapshot read before commit and is not re-validated at commit:

| Pattern | Result under N-way single-key contention |
|---|---|
| `addTo`/`subtractFrom` committed aggregate | **EXACT (safe)** — both engines |
| `ifVersion`-guarded write (CAS) | leaks (#1288) |
| create-only first-write-wins | leaks (#1288, F-006) |
| naive read-modify-write | lost updates |
| REST `If-Match` | silently ignored (no precondition on writes) |
| read-the-counter-back-to-decide | unreliable on both engines |

Consequence gaps customers hit:
- `addTo` returns void — no "read your serialized position" → can't build an exactly-once claim/limit synchronously.
- `subtractFrom` has no floor → inventory oversells to negative under concurrency, with no safe single-node guard.
- The transaction retry-on-conflict replays recorded writes WITHOUT re-running `post()`, so an app-level CAS-retry loop can't re-decide (livelocks).

This affects common customer patterns: rate limiters, webhook/claim queues, inventory decrement, idempotency stores, status state-machines.

## What DOES work (validated)
An asynchronous recipe achieves exactly-once on a single node (0-double/0-lost at 24-way): each claimant writes its OWN non-contended row, uses `addTo` only to COUNT, and a single-reader sweep over committed state picks the winner. Synchronous "claim exactly one in the claiming request" still requires external coordination / a true single-writer queue (`threads.count=1` does NOT serialize commits).

## Asks
1. **Docs:** the safe-concurrent-write matrix + guidance (commutative deltas only; serialize non-commutative transitions externally; the async claim recipe) — tracked at documentation#531.
2. **Product/design (this issue):** consider a commit-time atomic primitive — a true compare-and-set, a clamped/conditional decrement (floor at 0), and/or an atomic claim that returns a unique serialized position — to make these patterns correct without external coordination.

## Sources
Exploratory QA campaign (qa-explorer), scenarios QA-099/100/101/103/104/105/106; underlying primitive bug = #1288. Reference tests in `integrationTests/qa-scratch/` (`cas-audit`, `rate-limiter`, `inventory-oversell`, `claim-recipe`, `webhook-queue-sm`). Harper `001bf7b9c` (v5.1.0).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.