HarperFast / HarperFast/harper
A write from an aborted replicated transaction can commit on its own when another key's load fails
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 200
Description
## Summary
When a replicated transaction is applied and one record's resource load rejects, the transaction aborts — but a *different* key's write, still suspended on its own slow load, can resume afterwards and reach `addWrite()` on the now-closed transaction, where `save()` commits it **alone, with its own audit entry**. The follower durably applies part of a leader transaction that never landed as a whole.
Pre-existing. harper#2235 narrows it (same-key successors now short-circuit when their predecessor rejects) but cannot close it, because the surviving write is under a different key.
## Mechanism (confirmed in source)
The apply loop dispatches each record's `writeUpdate()` without awaiting it, and `writeUpdate()` suspends on `await Table.getResource(...)`. For a leader transaction `{put A, delete B}`:
1. A's `getResource` rejects. The rejection propagates through `Promise.all(writePromises)` at `end_txn` and the transaction aborts.
2. B's load was slow. Its continuation resumes after the abort and calls `_writeDelete` → `txnForContext(context)` → the context's transaction is closed.
3. In `DatabaseTransaction.save()`, a closed transaction takes the `immediateCommit` branch: a fresh `RocksTransaction` is created and committed on the spot. B lands, with its own audit entry.
## Two different outcomes depending on the write type
This is worth stating because the two are opposite failures and only one is the "escape":
- **Eagerly-saved write types** (`delete`, `invalidate`, `relocate`, `publish`) — `addWrite` calls `save()`, which commits them alone. **A write from an aborted transaction becomes durable.**
- **`put`/`patch`** — `deferSave: true`, so `addWrite` does not call `save()` and nothing else does either. The write is staged into a transaction nobody ever commits and is **silently discarded**. Observed while testing harper#2235: in that configuration neither write commits and no error surfaces.
So the same race either over-applies or under-applies, depending on which write survives.
## Impact
Follower diverges from leader: it holds a delete the leader's transaction rolled back, or drops a put that the leader committed. Nothing alarms — the apply loop logs the original failure and carries on, and the divergence persists until a full copy or a later write to that key rewrites it.
## Suggested fix
An `open !== TRANSACTION_STATE.OPEN` check in `writeUpdate`'s continuation, so a write whose transaction is already gone is dropped rather than committed alone. That is a small change, but it decides drop-vs-commit semantics for **every** source-applied write, so it wants its own tests — including the `put` case above, where the current behavior is already a silent drop.
Documented in `DESIGN.md` under the repeat-writes section as part of harper#2235.
Contributor guide
Research direction
Start at the replicated apply loop and follow writeUpdate through Table.getResource and DatabaseTransaction.save, using the cross-key failure sequence in the issue. Review DESIGN.md's repeat-writes section, then add regression coverage for surviving delete and put writes after another key's load rejects. Done means neither write commits outside the aborted transaction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, nodejs
- Domain
- backend, databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100