HarperFast / HarperFast/harper
[Design] No safe non-commutative concurrency primitive (compare-and-set / claim-once / clamped decrement) on a single node
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 200
Description
## Summary
A consolidation of the campaign's concurrency findings into one design/capability observation: **Harper v5 single-node has no safe non-commutative concurrent-write primitive.** The only primitive whose committed result is exact under single-key contention is the commutative atomic delta (`addTo`/`subtractFrom`). Every compare-and-set-shaped idiom is unsafe under concurrency (RocksDB rare/narrow, LMDB on every round), because each takes a snapshot read before commit and is not re-validated at commit:
| Pattern | Result under N-way single-key contention |
|---|---|
| `addTo`/`subtractFrom` committed aggregate | **EXACT (safe)** — both engines |
| `ifVersion`-guarded write (CAS) | leaks (#1288) |
| create-only first-write-wins | leaks (#1288, F-006) |
| naive read-modify-write | lost updates |
| REST `If-Match` | silently ignored (no precondition on writes) |
| read-the-counter-back-to-decide | unreliable on both engines |
Consequence gaps customers hit:
- `addTo` returns void — no "read your serialized position" → can't build an exactly-once claim/limit synchronously.
- `subtractFrom` has no floor → inventory oversells to negative under concurrency, with no safe single-node guard.
- The transaction retry-on-conflict replays recorded writes WITHOUT re-running `post()`, so an app-level CAS-retry loop can't re-decide (livelocks).
This affects common customer patterns: rate limiters, webhook/claim queues, inventory decrement, idempotency stores, status state-machines.
## What DOES work (validated)
An asynchronous recipe achieves exactly-once on a single node (0-double/0-lost at 24-way): each claimant writes its OWN non-contended row, uses `addTo` only to COUNT, and a single-reader sweep over committed state picks the winner. Synchronous "claim exactly one in the claiming request" still requires external coordination / a true single-writer queue (`threads.count=1` does NOT serialize commits).
## Asks
1. **Docs:** the safe-concurrent-write matrix + guidance (commutative deltas only; serialize non-commutative transitions externally; the async claim recipe) — tracked at documentation#531.
2. **Product/design (this issue):** consider a commit-time atomic primitive — a true compare-and-set, a clamped/conditional decrement (floor at 0), and/or an atomic claim that returns a unique serialized position — to make these patterns correct without external coordination.
## Sources
Exploratory QA campaign (qa-explorer), scenarios QA-099/100/101/103/104/105/106; underlying primitive bug = #1288. Reference tests in `integrationTests/qa-scratch/` (`cas-audit`, `rate-limiter`, `inventory-oversell`, `claim-recipe`, `webhook-queue-sm`). Harper `001bf7b9c` (v5.1.0).
Contributor guide
Assessment
This issue has not been assessed yet.