electric-sql / electric-sql/electric-circuits
DDL and TRUNCATE leave shapes silently stale (schema read once at boot)
- Dominant language
- Rust
- Stars
- 30
- Forks
- 6
- PR merge metrics
- No merged PRs in 30d
Description
Found during the production-readiness investigation.
**Symptoms / failure scenarios:**
- Schema is introspected once in `setup_postgres` and never refreshed. After `ALTER TABLE ... ADD COLUMN`, `parse_cols` silently drops the new column from every envelope (`apps/engine/src/replication.rs:447-449`) — synced rows are missing the column until an engine restart, with no error.
- `TRUNCATE` produces no envelopes — it only logs (`replication.rs:269-276`). Truncated rows linger **forever** in every shape, aggregate, and subquery inner set.
- A `REPLICA IDENTITY` regression (e.g. a migration recreated the table) degrades to forwarding new-only updates and skipping deletes (`replication.rs:231-265`) — shapes keep phantom rows.
In all three cases we keep serving wrong data quietly. Upstream invalidates affected shapes (409 → client refetch) and runs a 60s `SchemaReconciler` sweep to catch silent DDL.
**Fix direction:** periodic re-introspection diffed against the registered schema; on change, TRUNCATE, or replica-identity regression → invalidate affected shapes (drop + refetch signal), never degrade silently. (Moving to pgoutput later provides relation messages for free.)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Contributor guide
Research direction
Start with setup_postgres and replication.rs:231-276 and 447-449, then trace how registered schemas and shapes are represented. Compare the existing invalidation and refetch behavior described in the issue with the DDL, TRUNCATE, and replica-identity paths. Done means affected shapes are invalidated and refetched rather than silently serving stale or incomplete data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, rust
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100