No public barrier for "the fetched rows have been applied": `dataUpdatedAt` advances while the collection still holds the old rows, and an identical-payload refetch commits with zero observable effect
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 3.9k
- Forks
- 266
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 55
Description
Reproduction executed on set A (0.7.2 / 1.2.4); set B (0.9.2 / 1.2.15, the latest published)
was installed and verified by source reading, not re-run.
Versions. Two sets, both against @tanstack/query-core@5.102.8:
A = @tanstack/db 0.7.2 + query-db-collection 1.2.4 + react-db 0.2.1;
B = @tanstack/db 0.9.2 + query-db-collection 1.2.15 + react-db 0.4.1.
Being precise about what was run where: the reproduction below was executed on set A. Set B
was installed and its dist/esm read, but not re-run — so every claim about B in this report is
a source reading, and I have marked the line numbers accordingly. react-db is listed only for
completeness; none of this involves a React binding.
Summary
A consumer cannot ask a query collection "are the rows I can read right now the ones the last
successful fetch returned?". Two halves:
- Measured — an identical-payload refetch leaves no trace at all: no change events, no
_stateRevisionbump, no ready event, whiledataUpdatedAtdoes advance. - Read from source, not measured — the observer publishes before the collection commits.
On the async apply pathutils.dataUpdatedAtadvances andutils.fetchStatusreturns to
idlebeforeapplySuccessfulResultruns, soutilswould report a settled, successful,
non-fetching query over the previous rows.collection.statuscan't close the interval —
it is stickyready.
(1) is the concrete request. It also happens to be what would make (2) fixable in user land: the
natural workaround for (2) is a watermark — "trust the rows only once an apply newer than
dataUpdatedAt is observed" — and (1) makes that watermark latch permanently false after the
first unchanged refetch. utils.refetch() is not a barrier either; it resolves independently of
the commit.
(1) Reproduction — measured, self-contained, no persistence adapter
import { createCollection } from "@tanstack/db"
import { QueryClient } from "@tanstack/query-core"
import { queryCollectionOptions } from "@tanstack/query-db-collection"
import { it } from "vitest"
type Row = { id: string; n: number }
it("identical-payload refetch: dataUpdatedAt moves, nothing else does", async () => {
const queryClient = new QueryClient({ defaultOptions: { queries: { retry: false } } })
const collection = createCollection(queryCollectionOptions({
queryKey: ["repro"],
queryClient,
queryFn: async () => {
await new Promise((r) => setTimeout(r, 5))
return [{ id: "a", n: 1 }] satisfies Row[] // always identical
},
getKey: (row: Row) => row.id,
}))
await collection.preload()
let changeEvents = 0
const sub = collection.subscribeChanges(() => { changeEvents++ })
const rev = () => (collection as unknown as { _stateRevision: number })._stateRevision
const t0 = collection.utils.dataUpdatedAt
const r0 = rev()
for (let i = 1; i <= 3; i++) {
await new Promise((r) => setTimeout(r, 12)) // so the timestamps differ
await collection.utils.refetch()
await new Promise((r) => setTimeout(r, 12))
console.log(i, { dtUpdatedAt: collection.utils.dataUpdatedAt - t0, dRevision: rev() - r0,
changeEvents, status: collection.status })
}
sub.unsubscribe()
await collection.cleanup()
})
Actual — identical payload, then the same loop with n += 1 each round for contrast:
identical 1 { dtUpdatedAt: 22, dRevision: 0, changeEvents: 0, status: 'ready' }
identical 2 { dtUpdatedAt: 58, dRevision: 0, changeEvents: 0, status: 'ready' }
identical 3 { dtUpdatedAt: 101, dRevision: 0, changeEvents: 0, status: 'ready' }
changing 1 { dtUpdatedAt: 21, dRevision: 1, changeEvents: 1, status: 'ready' }
changing 2 { dtUpdatedAt: 55, dRevision: 2, changeEvents: 2, status: 'ready' }
changing 3 { dtUpdatedAt: 89, dRevision: 3, changeEvents: 3, status: 'ready' }
Expected: a public, monotonic signal that advances once per successful application of a query
result, whether or not the diff was empty, so a consumer can pair "rows I am reading" with "fetch
that produced them".
QueryCollectionUtilsImpl is identical in 1.2.4 and 1.2.15; its non-write members are
clearError, lastError, isError, errorCount, isFetching, isRefetching, isLoading,
dataUpdatedAt, fetchStatus (the class also carries refetch and
writeInsert/writeUpdate/writeDelete/writeUpsert/writeBatch). Nothing there reports
application.
(2) The ordering interval — read from source, NOT measured
Stating this plainly because it matters for how much weight to give it: I could not stage this
one. On query-db-collection@1.2.4 a plain query collection calls applySuccessfulResult
synchronously inside the observer callback, so there is no interval to observe there — confirmed
by measurement.
On 1.2.4 the interval is one-shot per query key. handleQueryResult branches at :670:
retainedQueriesPendingRevalidation.has(key) takes the async void reconcileSuccessfulResult(...)
path (:682), and everything else falls through to a plain synchronous
applySuccessfulResult(...) (:689) that lands rows in the same tick as the observer
notification. The set's only .add is :507 (gate :506, metadata lookup :503-505), reachable
only from createQueryFromOpts, and :644 deletes the key immediately after commit(). So the
interval opens for the first successful result of a rehydrated persisted query collection and
then closes for good. That is a narrow window, but it lands at app launch — which for us is
exactly when an irreversible decision reads the flag.
On 1.2.15 it is no longer one-shot. Both branches now go through enqueueResultApplication
(:829 retained, :839 otherwise) and applySuccessfulResult is itself async (:627), so the
synchronous fall-through is gone and every successful result applies asynchronously. If I'm
reading that right, the version that adds the apply-serialization layer is also the version that
widens this interval from once-per-key to every refetch — which seems worth a second opinion from
someone who knows the intent.
What I tried (all on 0.7.2 / 1.2.4), sampling from a queryCache.subscribe callback and from
a setTimeout(0) polling loop, looking for any sample with dataUpdatedAt advanced,
isFetching === false, isError === false and the row still at its old value:
- plain query collection — 0 such samples (expected: synchronous apply);
- persisted collection with a stub adapter — 0 such samples; the retained path was never armed,
becauseretainedQueriesPendingRevalidationis only populated when a persisted query-retention
entry already exists (dist/esm/query.js:503-508); - two sessions over one real
node:sqlite-backed adapter, to create that retention entry —
0 such samples, but inconclusive: session 2 never rehydrated the old row (it went
undefined → 2), so the precondition "collection holds the previous row" was never set up.
So: not reproduced, and not disproven either — my harness failed to establish the precondition on
the one path where the interval is reachable. In hindsight the branch structure above explains
each miss: the first two never armed retainedQueriesPendingRevalidation at all, and the third
armed it but never rehydrated a row that could go stale. If you think the reading below is wrong, that
would be useful to know; if it's right, a maintainer can stage it far more cheaply than I can. The
source path is short enough to point at directly:
| 1.2.4 | 1.2.15 | |
|---|---|---|
reconcileSuccessfulResult awaits baseline, then applies |
dist/esm/query.js:648-655 |
:745-747 |
applySuccessfulResult (begin→write→commit→markReady) |
:580-647 (commit :643, markReady :646) |
:627-731 (commit :720, markReady :731), now async |
| only differing rows are written | :614-641 |
same shape |
@tanstack/db |
0.7.2 | 0.9.2 |
|---|---|---|
markReady() no-ops once already ready |
collection/lifecycle.js:76-92 |
:87-120 |
empty ready event only on loading → ready |
collection/lifecycle.js:89-91 |
:113 |
stateRevision bumps only when changes.length > 0 |
collection/changes.js:43-44 |
:58-59 |
| empty batch returns before notifying subscribers | collection/changes.js:58-60 |
:126-129 (publishEvents) |
What 1.2.15 already has, and why it doesn't close this
1.2.15 adds a serialized apply-tracking layer that 1.2.4 has no trace of, and it is the closest
thing in the codebase to what this issue asks for — flagging it so this doesn't read as if it had
been missed:
pendingResultApplications/failedResultApplications/resultApplicationTokens/
resultApplicationControllers(dist/esm/query.js:223-226);getResultApplicationSettlement(hashedQueryKey)(:237-244) — returns the in-flight apply
promise, ortrueonce settled. That is exactly the predicate we want;enqueueResultApplication(:782-798), called synchronously fromhandleQueryResult
(:829,:839), chaining each apply onto the previous one;applySuccessfulResultis nowasync(:627) and awaited at:751, with the
applicationToken/signalpair at:745.
The reason it doesn't help a consumer: getResultApplicationSettlement is consumed only inside
createQueryFromOpts (:554, :558, :563, :612, :616, :623), which is reachable from
eager sync start (:908) and from the non-eager loadSubsetDedupe (:1087). It is never
consulted per-refetch, and it is not reachable through collection.utils. So it serializes
applies against each other, but does not give a consumer anything to await or observe.
If widening that settlement to the refetch path and exposing it is easier than either option
below, that would work just as well for us.
The ask
The measured half alone is enough to motivate this. Either shape would work; the first looks like
the smaller change:
- A "sync applied" signal that fires on every commit, including empty ones — bump
stateRevision(or a siblingsyncRevision) on everycommit()regardless of diff size, and
expose it publicly. - An awaitable publication barrier —
utils.refetch()resolving only after
applySuccessfulResulthas committed, or acollection.whenApplied()promise.
The property we need either way: for every successful fetch, exactly one observable event, even
when the resulting diff is empty.
Related, not the same
#661 (awaitPersisted()) awaits mutation persistence by key (client → server); this is the
sync direction, server rows → collection. #1657 / #1659 are adjacent but a different concern.
We also see a lost write with serialized whole-row updates on 0.8/0.9 over a persisted collection
(the optimistic delta drops at transaction settle, before the canonical row is published) — likely
downstream of the same window, but a distinct symptom with its own reproduction, so it is kept out
of this report and can be filed separately.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Vitest reproduction in the issue, then inspect query-db-collection's dist/esm/query.js around handleQueryResult, enqueueResultApplication, and applySuccessfulResult. Compare its commit path with @tanstack/db's collection/changes.js and lifecycle.js, especially empty batches and stateRevision. Done means a public signal or barrier advances exactly once after every successful application, including identical payloads, and the reproduction observes it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- sqlite, typescript
- Domain
- database, developer-experience, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100