fly-apps / fly-apps/safe-ecto-migrations

adding generated stored columns on large tables in Postgres

Open
#17 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
387
Forks
17
PR merge metrics
No merged PRs in 30d

Description

reference: https://stackoverflow.com/questions/77852268/how-to-add-a-stored-generated-column-to-a-very-large-table

tldr, when adding a generated stored column to an existing large table, it will lock the table so it can calculate the value for each row.

There seems to be a workaround:

1. Add the column as a normal nullable column
2. use a trigger `BEFORE INSERT OR UPDATE` with a function that is equivalent to what you would put as the as generated expression.
3. Backfill to fill the column
4. In Postgres 17, it may be possible to alter the column to set an expression, allowing you to drop the trigger.

However, there seems to be a trade-off in that INSERT times are slower with a triggered function. source (4yrs old, for postgres v12): https://www.ongres.com/blog/generate_columns_vs_triggers/

```sql
CREATE OR REPLACE FUNCTION generate_foo_immutable ()
RETURNS TRIGGER
AS $$
BEGIN
NEW.foo = NEW.bar * 2;
RETURN new;
END;
$$
LANGUAGE plpgsql IMMUTABLE;

CREATE TRIGGER generate_foo_immutable_trigger
BEFORE INSERT OR UPDATE [OF other_column1, other_column2, ...] ON foo_table
FOR EACH ROW
EXECUTE PROCEDURE public.generate_foo_immutable();
```

Contributor guide

No contributing guide indexed for this repository

Research direction

No project files, tests, or entry points are named. Start by reading the linked PostgreSQL discussion and comparing it with the safe migration behavior described here; the intended implementation scope and completion criteria still need to be defined.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgres
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.