Persisted collections: sync transaction straddling a hydrate-window close is buffered but never flushed (lost update)
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 3.9k
- Forks
- 266
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 55
Description
@tanstack/db-sqlite-persistence-core 0.2.12 (persisted.ts), with @tanstack/db 0.7.2 + @tanstack/electric-db-collection 0.3.18 over @tanstack/browser-db-sqlite-persistence.
The race: a sync transaction captures queuedBecauseHydrating at begin() but is only pushed into queuedHydrationTransactions at commit(). hydrateSubsetUnsafe clears isHydrating and then flushes the buffer. If the hydrate window closes between the transaction's begin() and commit(), the flush runs first and finds the buffer empty, the commit pushes afterwards, and nothing ever drains that entry again — the transaction is dropped for the life of the collection. The client sits on stale data while the rows sit in its own buffer.
The straddle is easy to hit because the Electric adapter holds one sync transaction open across two fetch responses (schema → rows, ~16 ms apart in our traces), so it races OPFS loadSubset resolution directly. Captured live (page-time ms, one collection):
p=4983 hydrate.enter hid=3
p=4998 fetch.res id=1 <- begin(), inside the window
p=5005 hydrate.exit hid=3 buffered:0 <- flush finds nothing
p=5015 fetch.res id=2 <- rows arrive, commit()
p=5017 tx.buffered ops=120 <- parked 12 ms too late; never applied
In our app the fast path (a small write-behind adapter) made disk hydrates complete in single-digit ms, which flips this race often enough that a two-tab Playwright journey fails ~1 in 8 under CPU load — the "colleague" tab never renders an update that provably arrived on its own wire. With slower hydrates the same race instead surfaces as multi-second delivery delays, so it is easy to misattribute.
Fix that verified for us — drain immediately when a transaction is buffered after the window already closed:
queueHydrationBufferedTransaction(
transaction: BufferedSyncTransaction<T, TKey>,
): void {
this.queuedHydrationTransactions.push(transaction)
+ // `queuedBecauseHydrating` is captured at begin() but the buffer is only
+ // filled at commit(). If the hydrate window closed in between, its flush
+ // already ran and found the buffer empty — drain the late entry here or
+ // it is dropped for the life of the collection.
+ if (!this.isHydrating) {
+ void this.applyMutex
+ .run(() => this.flushQueuedHydrationTransactionsUnsafe())
+ .catch((error: unknown) => {
+ console.warn(
+ `Failed to flush a late hydration-buffered transaction:`,
+ error,
+ )
+ })
+ }
}
Verified with a deterministic harness (force-marking the first sync tx per collection as begun-while-hydrating): without the drain, 18/18 forced transactions orphaned and the journey fails; with it, 32/32 applied, 0 orphaned, and the journey is green 9/9 including under CPU load. We are carrying this as a pnpm patch; happy to send it as a PR.
Related to our storm report #1752 — the two compound: long hydrate windows raise the buffering rate, and this race turns a buffered transaction into a lost one.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in persisted.ts at queueHydrationBufferedTransaction and hydrateSubsetUnsafe, then trace how queuedHydrationTransactions is flushed around the hydrate window. Reproduce the deterministic harness described in the issue, including the two-tab journey under CPU load. Done means transactions that commit after the window closes are applied without orphaned entries.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- sqlite, typescript
- Domain
- database
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100