HarperFast / HarperFast/harper
Live subscription can deliver a CREATE audit record as `type: "delete"` when the entry is already gone
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 200
Description
## Summary
The live subscription listener can deliver a record's **CREATE** audit record to subscribers as `type: "delete"`, stamped with the create's own version. This is a wrong event rather than a missing one, and it reaches all three live surfaces (MQTT, SSE, WebSocket).
## Mechanism (verified on `origin/main` @ `fb762a365`)
The per-key listener registered via `addSubscription` (from `resources/Table.ts` `subscribe()`; listener body at `Table.ts:4230-4265`) does not trust `auditRecord.type`. For any non-`message`/`reload`/`end_txn` audit record it recomputes the outgoing type from the **current** primary-store state:
```js
const entry: Entry = primaryStore.getEntry(id);
if (entry) {
if (entry.version !== (auditRecord.recordVersion ?? auditRecord.version)) return;
value = entry.value;
type = entry.metadataFlags & INVALIDATED ? 'invalidate' : value ? 'put' : 'delete';
} else {
type = 'delete'; // Table.ts:4255
}
event.version = auditRecord.version; // Table.ts:4262 — the ORIGINAL record's version
```
If `entry` is missing at delivery time — because the record was deleted or evicted after the CREATE committed but before that audit record's turn came up — a `put` (create) audit record is delivered as `type: "delete"` while carrying the create's version.
The window is real, not theoretical: dispatch comes from a batched, deferred audit-log tail. `resources/transactionBroadcast.ts` `notifyFromTransactionData` uses `NOTIFY_BATCH_SIZE = 256` (:138), defers via `setImmediate` (:58), and calls `subscription.listener(recordId, auditRecord, timestamp, beginTxn)` at :222 — so a fast subsequent delete or a TTL eviction can already have removed the entry by the time the create's callback fires.
## Consequence
A subscriber doing event-sourced replay never sees the create and instead sees a delete carrying the create's version — so ordering and identity are both wrong, not merely incomplete. Create-then-fast-delete and create-under-TTL are ordinary workloads, so the trigger is structural rather than exotic.
## Confirming experiment (for whoever picks this up)
Instrument `Table.ts:4248` (`primaryStore.getEntry(id)`) to log when it returns falsy inside this listener, alongside `auditRecord.type`. Then drive create-then-immediate-delete (or create-then-TTL-evict) on the same key under an open collection subscription and watch for `auditRecord.type === 'put'` reaching the `else { type = 'delete' }` branch. That log line, correlated with the delivered event's version equalling the create's version, distinguishes this misclassification from a coincidental legitimate delete.
## Related
Sibling in the same file family: #2311 (subscription replay silently drops events committed after the audit cursor terminates) — that one is a *silent drop* in a different branch; this one is a *wrong event*.
*Found by the qa-explorer QA loop (finding QA-675, flagged there at MEDIUM confidence and documented in the spec rather than asserted); mechanism since confirmed from source. Dup-searched in `harper` and `harper-pro` — nothing tracks it.*
Contributor guide
Research direction
Start in resources/Table.ts at the addSubscription listener around lines 4230-4265, then read resources/transactionBroadcast.ts around notifyFromTransactionData and the deferred batch dispatch. Confirm the race with the create-then-immediate-delete or TTL experiment described in the issue. Done means a CREATE audit record is no longer delivered as a delete, with correct event identity and version across MQTT, SSE, and WebSocket subscriptions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, nodejs
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100