HarperFast / HarperFast/harper
Optimistic concurrency (ifVersion) conflict-detection leaks under concurrent same-key writes (LMDB severe, RocksDB mild)
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 200
Description
## Summary
A consolidated characterization (qa-explorer) of several concurrency findings: the `ifVersion` optimistic conflict-detection path does not reliably serialize concurrent same-key **version-guarded snapshot-read-then-write** operations. `addTo` (commit-time-resolved atomic) is correct everywhere; the version-guarded classes leak — LMDB catastrophically, RocksDB mildly.
## The matrix (harper `001bf7b9c`, qa-scratch/lmdb-concurrency)
N concurrent ops on the SAME key, anomaly = lost-updates / duplicate-winners:
| Op class | RocksDB | LMDB |
|---|---|---|
| **`addTo` (atomic)** | **0 lost — correct** | **0 lost — correct** |
| Create-only (`insert`/`create()`) — expect exactly 1 | occasional dup (1-2 created @10/30) | **always leaks (3-4 created)** |
| Conditional transition (`ifVersion` guard) — expect 1 winner | rare double @100 | **always leaks (3-5 winners)** |
| Unguarded full-put RMW | lost updates (likely expected LWW) | lost updates (worse) |
## Interpretation
- **`addTo` is the safe primitive** for counters/accumulators — recommend it in docs.
- Create-only exactly-once and conditional-transition single-winner are violated under concurrency — these are real correctness bugs, severe on LMDB.
- Root cause appears to be the **`ifVersion` conflict-detection** for version-guarded writes (e.g. create-lock `ifVersion(null)` at Table.js:1612), not a single engine — LMDB just exposes it badly.
- (Unguarded full-put 'lost updates' is likely expected last-write-wins; NOT claimed as a defect here.)
## Related
- #1283 (atomic increment ⨯ TTL-expiry race) is a specific instance of this family.
- Practical impact: idempotency/dedup stores and 'reserve last unit' patterns are unsafe under concurrency, especially on LMDB.
## Repro
`npm run test:integration -- "integrationTests/qa-scratch/lmdb-concurrency.test.ts"` (and with `HARPER_STORAGE_ENGINE=lmdb`). Scratch matrix test ready to promote as the regression guard for the fix.
_Filed from qa-explorer (F-006 + ifVersion META, consolidating QA-001/004/021/025)._
Contributor guide
Assessment
This issue has not been assessed yet.