on-demand: every loadSubset forces an extra same-offset chunk request, tripping the Electric client's fast-retry-loop guard
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 68/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- postgres, typescript
- Domain
- databases
Research direction
Start at the on-demand Electric collection's loadSubset path and inspect its calls to stream.forceDisconnectAndRefresh() and requestSnapshot. Run the linked repro.mjs against the real Postgres and Electric setup, then verify that ten sequential subset loads produce ten subset requests without extra same-offset non-subset chunk requests or the fast-retry-loop warning.
Written by the indexing model from the issue text.
Description
- I've validated the bug against the latest version of DB packages (
@tanstack/db@0.9.2,@tanstack/electric-db-collection@0.4.10,@electric-sql/client@1.5.28)
Describe the bug
Every loadSubset on an on-demand Electric collection issues an extra, non-subset chunk request at the same offset, because each one calls stream.forceDisconnectAndRefresh() before requestSnapshot.
Against a log that is not changing, those extra requests are byte-identical and repeat the same offset — which is precisely the pattern @electric-sql/client treats as a broken cache or proxy. Ten ordinary sequential subset loads are enough to trip its fast-retry-loop guard, which clears the client's caches, resets the stream, and tells the developer to go and check their proxy configuration.
The proxy is fine. The duplicate requests come from the adapter.
To Reproduce
Runnable repro against a real Postgres + Electric (the pair this repo's own e2e suite expects), so nothing fakes the wire: https://gist.github.com/MAST1999/630589ae5beb96ce5c9222a5aad003b7
docker compose up -d
npm install
node repro.mjs
It creates ten live queries with distinct where predicates, each fully awaited before the next is created. No concurrency, nothing superseded, nothing racing, and the table never changes.
distinct subsets requested : 10
non-live shape requests : 20
snapshots (POST) : 0
chunk refetches (GET) : 20
distinct offsets requested : 4
requests per subset : 2.00
subset requests : 10
The requests alternate — ten are the subset requests themselves, and ten carry no subset__ parameters at all and repeat the same offset=0_inf:
GET ?log=changes_only&offset=now&subset__params=...&subset__where="n" >= $1&table=items
GET ?handle=...&log=changes_only&offset=0_inf&table=items
GET ?handle=...&log=changes_only&offset=0_inf&subset__params=...&subset__where=...
GET ?handle=...&log=changes_only&offset=0_inf&table=items
and the client's guard fires:
[Electric] Detected fast retry loop (5 requests in 500ms at the same offset).
Clearing client-side caches and resetting stream to recover. If this persists,
check that your proxy includes all query parameters (especially 'handle' and
'offset') in its cache key, and that required Electric headers are forwarded to
the client.
Expected behavior
Loading ten distinct subsets should cost ten subset requests. A refresh that is
only needed when the stream is actually behind should not be issued
unconditionally per subset, and ordinary usage should not trip a guard whose
message tells the developer to debug infrastructure that is working correctly.
Scope — what this repro does and does not show
It shows the request amplification (2.00 non-live requests per subset) and the spurious guard trip, both on the latest published versions.
It does not show a permanently dead stream — the client recovers in this repro. I've seen a stuck paused stream downstream of this in an application, but I could not reduce that to a minimal case, so I'm deliberately not claiming it here.
Additional context
@tanstack/db@0.9.0 removed the subset-algebra helpers (isLoadSubsetRequestSubsumedBy and friends) in favour of DeduplicatedLoadSubset sharing only exact demand identities, so a narrower predicate is no longer recognised as already covered. That makes distinct-but-overlapping predicates more common in practice, and each one now carries this extra refresh.
Related but distinct: #1855 and #1856.
- Dominant language
- TypeScript
- Stars
- 3.9k
- Forks
- 266
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 56
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from TanStack/db
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Index suggestion for collection size is gated on autoIndex, so it only fires where it is redundant Open
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
area:tools bug good first issue help wanted priority:P2
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
TaewoooPark/Motifcode#14 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
newrelic-experimental/preflight#793 · 1 comment ·
-
bug 🐞
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
[Bounty proposal] fix(web): memory insights count an evening memory on the next day ($25 proposed) Open
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
BasedHardware/omi#15320 ·