electric-sql / electric-sql/electric
Shape not invalidated when column nullability changes (ALTER COLUMN SET/DROP NOT NULL)
- Dominant language
- TypeScript
- Stars
- 10.4k
- Forks
- 375
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 18
Description
## Problem
When a column's nullability changes via `ALTER COLUMN ... DROP NOT NULL` or `SET NOT NULL`, Electric SQL does not detect the schema change and existing shapes continue serving data with stale column metadata.
This causes `ParserNullValueError` on the client side when a previously-NOT-NULL column receives NULL values after being made nullable.
## Root cause
This is fundamentally a PostgreSQL limitation: the logical replication protocol's `Relation` message only includes column `name`, `type_oid`, `type_modifier`, and `flags` (key membership). **Nullability is not part of the Relation message.**
As a result:
1. `ALTER COLUMN ... DROP NOT NULL` doesn't change any field in the Relation message
2. PostgreSQL doesn't emit a new Relation message at all
3. `ShapeLogCollector.AffectedColumns.find_differing_columns/2` is never called
4. Even if it were called, `Changes.Column` only has `name` and `type_oid` — no nullability to compare
## What works vs what doesn't
| Schema change | Relation message emitted? | Shape invalidated? |
|--------------|---------------------------|-------------------|
| ADD COLUMN | Yes (new column) | Yes |
| DROP COLUMN | Yes (column removed) | Yes |
| Change column type | Yes (different type_oid) | Yes |
| **Change nullability** | **No** | **No** |
## Reproduction
1. Create a shape on a table with a NOT NULL column
2. `ALTER TABLE t ALTER COLUMN c DROP NOT NULL;`
3. `INSERT INTO t (c) VALUES (NULL);`
4. Client receives NULL for column `c` → `ParserNullValueError`
## Workaround
- Restart Electric SQL to clear all shape caches
- Set `ELECTRIC_ENABLE_INTEGRATION_TESTING=true` to enable the `DELETE /v1/shape` API (but this env var name is not intended for production use)
Contributor guide
Assessment
This issue has not been assessed yet.