electric-sql / electric-sql/electric

Shape not invalidated when column nullability changes (ALTER COLUMN SET/DROP NOT NULL)

Open
#4,074 0 comments 0 reactions 0 assignees View on GitHub
claude
Dominant language
TypeScript
Stars
10.4k
Forks
375
Avg merge
3d 1h
Merged PRs (30d)
18

Description

## Problem

When a column's nullability changes via `ALTER COLUMN ... DROP NOT NULL` or `SET NOT NULL`, Electric SQL does not detect the schema change and existing shapes continue serving data with stale column metadata.

This causes `ParserNullValueError` on the client side when a previously-NOT-NULL column receives NULL values after being made nullable.

## Root cause

This is fundamentally a PostgreSQL limitation: the logical replication protocol's `Relation` message only includes column `name`, `type_oid`, `type_modifier`, and `flags` (key membership). **Nullability is not part of the Relation message.**

As a result:
1. `ALTER COLUMN ... DROP NOT NULL` doesn't change any field in the Relation message
2. PostgreSQL doesn't emit a new Relation message at all
3. `ShapeLogCollector.AffectedColumns.find_differing_columns/2` is never called
4. Even if it were called, `Changes.Column` only has `name` and `type_oid` — no nullability to compare

## What works vs what doesn't

| Schema change | Relation message emitted? | Shape invalidated? |
|--------------|---------------------------|-------------------|
| ADD COLUMN | Yes (new column) | Yes |
| DROP COLUMN | Yes (column removed) | Yes |
| Change column type | Yes (different type_oid) | Yes |
| **Change nullability** | **No** | **No** |

## Reproduction

1. Create a shape on a table with a NOT NULL column
2. `ALTER TABLE t ALTER COLUMN c DROP NOT NULL;`
3. `INSERT INTO t (c) VALUES (NULL);`
4. Client receives NULL for column `c` → `ParserNullValueError`

## Workaround

- Restart Electric SQL to clear all shape caches
- Set `ELECTRIC_ENABLE_INTEGRATION_TESTING=true` to enable the `DELETE /v1/shape` API (but this env var name is not intended for production use)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.