TanStack / TanStack/db

Persisted Electric collection ends up permanently empty: diverging schemaVersions wipe rows, schema reset leaves the resume point behind

Open
#1,589 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
3.9k
Forks
266
Avg merge
1d 4h
Merged PRs (30d)
55

Description

Packages: @tanstack/browser-db-sqlite-persistence 0.2.0, @tanstack/db-sqlite-persistence-core 0.2.0, @tanstack/electric-db-collection 0.3.6, @tanstack/db 0.6.8

We hit a case where a persisted Electric collection comes up ready with 0 rows on every page load, with nothing logged, and never recovers. It took a while to track down because it looked random. There seem to be two separate bugs that line up to cause it.

Diverging schemaVersions wipe rows through a shared coordinator adapter slot

createBrowserWASQLitePersistence caches one adapter per (schemaMismatchPolicy, schemaVersion) and calls coordinator.setAdapter(adapter) on every resolvePersistenceForCollection (see browser-persistence.js). BrowserCollectionCoordinator only has a single adapter slot, so whichever collection resolved last owns it.

If you have two persisted collections with different schemaVersions, leader-side RPCs (pullSince, applyLocalMutations, ensureRemoteSubset) for collection A can end up running through collection B's adapter. SQLiteCorePersistenceAdapter.ensureCollectionReady then compares A's registry schema_version against the other adapter's version, sees a mismatch, and under sync-present-reset deletes all of A's rows and rewrites the registry version. With both collections live the wipes flip-flop depending on RPC timing, which is why it looked random for us (the difference between F5 and a hard reload was pure timing).

A schema-mismatch reset doesn't clear collection_metadata, so Electric's resume point survives the wipe

The reset deletes rows, tombstones and applied_tx, but leaves collection_metadata intact. @tanstack/electric-db-collection keeps its resume point there (electric:resume, offset + handle, keyed by shape identity = url + params). So on the next load the collection restores 0 rows, reads the stale resume point, resumes the stream "up to date" past all the data, and marks itself ready. Nothing triggers a refetch, and the session even rewrites the resume record, so it stays empty across reloads.

Reproduction

const persistence = createBrowserWASQLitePersistence({ database, coordinator });

const a = createCollection(persistedCollectionOptions({
  ...electricCollectionOptions({ id: "a", shapeOptions: { url: shapeA }, getKey: r => r.id }),
  persistence, schemaVersion: 2,
}));
const b = createCollection(persistedCollectionOptions({
  ...electricCollectionOptions({ id: "b", shapeOptions: { url: shapeB }, getKey: r => r.id }),
  persistence, schemaVersion: 1, // different version
}));
  1. Sync both collections (put a few thousand rows in one to make it obvious) and confirm rows are persisted.
  2. Reload a few times. Depending on which adapter holds the coordinator slot when an RPC fires, one collection's table gets wiped while its electric:resume survives.
  3. After that it loads ready with 0 rows on every reload, nothing logged. scanRows returns 0 rows, but loadCollectionMetadata still shows a kind: "resume" record.

Workaround

Using one shared schemaVersion for every collection on the same persistence avoids the cross-wiring. We also bake that version into the shape URL, so bumping it changes Electric's shape identity and discards the stale resume point too.

Related issues

  • #1419 (closed) used the same diverging-schemaVersion setup (todos schemaVersion: 2, projects schemaVersion: 1) on a shared db/coordinator. It was closed by recommending a single shared persistence instance to fix a transaction-conflict error, but that doesn't help here: the coordinator still has one adapter slot, so diverging schemaVersions on a shared persistence still cross-wire and wipe.
  • #1478 / #1493 are about rows being wiped while the resume point survives, but via the progressive atomic-swap truncate. The second bug here is a different trigger (the schema-mismatch reset not clearing collection_metadata) ending in the same empty state.
  • #1443 is Electric + coordinator never reaching ready. This is the opposite: it reaches ready, just with 0 rows.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with browser-persistence.js, BrowserCollectionCoordinator, and SQLiteCorePersistenceAdapter.ensureCollectionReady. Reproduce the two-collection case with differing schemaVersion values, then inspect how schema-mismatch reset handles collection_metadata and the electric:resume record. Done means collections no longer cross-wire adapters and a reset cannot preserve a resume point for deleted rows.

Written by the indexing model from the issue text.

Assessment

Tech stack
sqlite, typescript
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.