electric-sql / electric-sql/electric-circuits

DDL and TRUNCATE leave shapes silently stale (schema read once at boot)

Open
#7 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Rust
Stars
30
Forks
6
PR merge metrics
No merged PRs in 30d

Description

Found during the production-readiness investigation.

**Symptoms / failure scenarios:**
- Schema is introspected once in `setup_postgres` and never refreshed. After `ALTER TABLE ... ADD COLUMN`, `parse_cols` silently drops the new column from every envelope (`apps/engine/src/replication.rs:447-449`) — synced rows are missing the column until an engine restart, with no error.
- `TRUNCATE` produces no envelopes — it only logs (`replication.rs:269-276`). Truncated rows linger **forever** in every shape, aggregate, and subquery inner set.
- A `REPLICA IDENTITY` regression (e.g. a migration recreated the table) degrades to forwarding new-only updates and skipping deletes (`replication.rs:231-265`) — shapes keep phantom rows.

In all three cases we keep serving wrong data quietly. Upstream invalidates affected shapes (409 → client refetch) and runs a 60s `SchemaReconciler` sweep to catch silent DDL.

**Fix direction:** periodic re-introspection diffed against the registered schema; on change, TRUNCATE, or replica-identity regression → invalidate affected shapes (drop + refetch signal), never degrade silently. (Moving to pgoutput later provides relation messages for free.)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Contributor guide

Open the contributing guide

Research direction

Start with setup_postgres and replication.rs:231-276 and 447-449, then trace how registered schemas and shapes are represented. Compare the existing invalidation and refetch behavior described in the issue with the DDL, TRUNCATE, and replica-identity paths. Done means affected shapes are invalidated and refetched rather than silently serving stale or incomplete data.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, rust
Domain
databases
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.