on-demand: every loadSubset forces an extra same-offset chunk request, tripping the Electric client's fast-retry-loop guard

Open
#1,857 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
68/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
postgres, typescript
Domain
databases

Research direction

Start at the on-demand Electric collection's loadSubset path and inspect its calls to stream.forceDisconnectAndRefresh() and requestSnapshot. Run the linked repro.mjs against the real Postgres and Electric setup, then verify that ten sequential subset loads produce ten subset requests without extra same-offset non-subset chunk requests or the fast-retry-loop warning.

Written by the indexing model from the issue text.

Description

  • I've validated the bug against the latest version of DB packages (@tanstack/db@0.9.2, @tanstack/electric-db-collection@0.4.10, @electric-sql/client@1.5.28)

Describe the bug

Every loadSubset on an on-demand Electric collection issues an extra, non-subset chunk request at the same offset, because each one calls stream.forceDisconnectAndRefresh() before requestSnapshot.

Against a log that is not changing, those extra requests are byte-identical and repeat the same offset — which is precisely the pattern @electric-sql/client treats as a broken cache or proxy. Ten ordinary sequential subset loads are enough to trip its fast-retry-loop guard, which clears the client's caches, resets the stream, and tells the developer to go and check their proxy configuration.

The proxy is fine. The duplicate requests come from the adapter.

To Reproduce

Runnable repro against a real Postgres + Electric (the pair this repo's own e2e suite expects), so nothing fakes the wire: https://gist.github.com/MAST1999/630589ae5beb96ce5c9222a5aad003b7

docker compose up -d
npm install
node repro.mjs

It creates ten live queries with distinct where predicates, each fully awaited before the next is created. No concurrency, nothing superseded, nothing racing, and the table never changes.

distinct subsets requested   : 10
non-live shape requests      : 20
  snapshots (POST)           : 0
  chunk refetches (GET)      : 20
distinct offsets requested   : 4
requests per subset          : 2.00
subset requests              : 10

The requests alternate — ten are the subset requests themselves, and ten carry no subset__ parameters at all and repeat the same offset=0_inf:

GET ?log=changes_only&offset=now&subset__params=...&subset__where="n" >= $1&table=items
GET ?handle=...&log=changes_only&offset=0_inf&table=items
GET ?handle=...&log=changes_only&offset=0_inf&subset__params=...&subset__where=...
GET ?handle=...&log=changes_only&offset=0_inf&table=items

and the client's guard fires:

[Electric] Detected fast retry loop (5 requests in 500ms at the same offset).
Clearing client-side caches and resetting stream to recover. If this persists,
check that your proxy includes all query parameters (especially 'handle' and
'offset') in its cache key, and that required Electric headers are forwarded to
the client.

Expected behavior

Loading ten distinct subsets should cost ten subset requests. A refresh that is
only needed when the stream is actually behind should not be issued
unconditionally per subset, and ordinary usage should not trip a guard whose
message tells the developer to debug infrastructure that is working correctly.

Scope — what this repro does and does not show

It shows the request amplification (2.00 non-live requests per subset) and the spurious guard trip, both on the latest published versions.

It does not show a permanently dead stream — the client recovers in this repro. I've seen a stuck paused stream downstream of this in an application, but I could not reduce that to a minimal case, so I'm deliberately not claiming it here.

Additional context

@tanstack/db@0.9.0 removed the subset-algebra helpers (isLoadSubsetRequestSubsumedBy and friends) in favour of DeduplicatedLoadSubset sharing only exact demand identities, so a narrower predicate is no longer recognised as already covered. That makes distinct-but-overlapping predicates more common in practice, and each one now carries this extra refresh.

Related but distinct: #1855 and #1856.

Dominant language
TypeScript
Stars
3.9k
Forks
266
Avg merge
1d 4h
Merged PRs (30d)
56

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from TanStack/db

All issues in TanStack/db

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.