Persisted Electric collection ends up permanently empty: diverging schemaVersions wipe rows, schema reset leaves the resume point behind
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 3.9k
- Forks
- 266
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 55
Description
Packages: @tanstack/browser-db-sqlite-persistence 0.2.0, @tanstack/db-sqlite-persistence-core 0.2.0, @tanstack/electric-db-collection 0.3.6, @tanstack/db 0.6.8
We hit a case where a persisted Electric collection comes up ready with 0 rows on every page load, with nothing logged, and never recovers. It took a while to track down because it looked random. There seem to be two separate bugs that line up to cause it.
Diverging schemaVersions wipe rows through a shared coordinator adapter slot
createBrowserWASQLitePersistence caches one adapter per (schemaMismatchPolicy, schemaVersion) and calls coordinator.setAdapter(adapter) on every resolvePersistenceForCollection (see browser-persistence.js). BrowserCollectionCoordinator only has a single adapter slot, so whichever collection resolved last owns it.
If you have two persisted collections with different schemaVersions, leader-side RPCs (pullSince, applyLocalMutations, ensureRemoteSubset) for collection A can end up running through collection B's adapter. SQLiteCorePersistenceAdapter.ensureCollectionReady then compares A's registry schema_version against the other adapter's version, sees a mismatch, and under sync-present-reset deletes all of A's rows and rewrites the registry version. With both collections live the wipes flip-flop depending on RPC timing, which is why it looked random for us (the difference between F5 and a hard reload was pure timing).
A schema-mismatch reset doesn't clear collection_metadata, so Electric's resume point survives the wipe
The reset deletes rows, tombstones and applied_tx, but leaves collection_metadata intact. @tanstack/electric-db-collection keeps its resume point there (electric:resume, offset + handle, keyed by shape identity = url + params). So on the next load the collection restores 0 rows, reads the stale resume point, resumes the stream "up to date" past all the data, and marks itself ready. Nothing triggers a refetch, and the session even rewrites the resume record, so it stays empty across reloads.
Reproduction
const persistence = createBrowserWASQLitePersistence({ database, coordinator });
const a = createCollection(persistedCollectionOptions({
...electricCollectionOptions({ id: "a", shapeOptions: { url: shapeA }, getKey: r => r.id }),
persistence, schemaVersion: 2,
}));
const b = createCollection(persistedCollectionOptions({
...electricCollectionOptions({ id: "b", shapeOptions: { url: shapeB }, getKey: r => r.id }),
persistence, schemaVersion: 1, // different version
}));
- Sync both collections (put a few thousand rows in one to make it obvious) and confirm rows are persisted.
- Reload a few times. Depending on which adapter holds the coordinator slot when an RPC fires, one collection's table gets wiped while its
electric:resumesurvives. - After that it loads ready with 0 rows on every reload, nothing logged.
scanRowsreturns 0 rows, butloadCollectionMetadatastill shows akind: "resume"record.
Workaround
Using one shared schemaVersion for every collection on the same persistence avoids the cross-wiring. We also bake that version into the shape URL, so bumping it changes Electric's shape identity and discards the stale resume point too.
Related issues
- #1419 (closed) used the same diverging-schemaVersion setup (todos
schemaVersion: 2, projectsschemaVersion: 1) on a shared db/coordinator. It was closed by recommending a single shared persistence instance to fix a transaction-conflict error, but that doesn't help here: the coordinator still has one adapter slot, so divergingschemaVersions on a shared persistence still cross-wire and wipe. - #1478 / #1493 are about rows being wiped while the resume point survives, but via the progressive atomic-swap truncate. The second bug here is a different trigger (the schema-mismatch reset not clearing
collection_metadata) ending in the same empty state. - #1443 is Electric + coordinator never reaching ready. This is the opposite: it reaches ready, just with 0 rows.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with browser-persistence.js, BrowserCollectionCoordinator, and SQLiteCorePersistenceAdapter.ensureCollectionReady. Reproduce the two-collection case with differing schemaVersion values, then inspect how schema-mismatch reset handles collection_metadata and the electric:resume record. Done means collections no longer cross-wire adapters and a reset cannot preserve a resume point for deleted rows.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- sqlite, typescript
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100