sqlalchemy / sqlalchemy/alembic

Adding a pseudo MigrationOp to add autocommit context wrapper to generated revision

Open
#1,825 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

autogenerate - rendering execution model use case
Dominant language
Python
Stars
4.4k
Forks
375
PR merge metrics
No merged PRs in 30d

Description

Describe the use case

We're using a PostgreSQL database and Squawk as linter in CI to minimise the risks of breaking production when applying migrations.
Among the checks, Squawk recommends creating or dropping indexes with the CONCURRENTLY parameter, however this operation cannot be performed in transactionnal DDL, which means we need to amend the generated migration script to add with op.get_context().autocommit_block(): around the operation
Manually updating the generated script is error prone and often detected after a couple round trips of failing CI.

I have written a pseudo MigrationOp called AutoCommitBlockOp, and I would like to share it with the community.
See example use below for more code

Databases / Backends / Drivers targeted

PostgreSQL (probably others, autocommit block is not restricted to a dialect)

Example Use

class AutoCommitBlockOp(ops.MigrateOperation):
    """Wraps a list of ops inside a with op.get_context().autocommit_block(): block.

    Use this for any DDL that PostgreSQL requires to run outside a transaction
    (e.g. CREATE INDEX CONCURRENTLY).
    """

    def __init__(self, nested_ops: list[ops.MigrateOperation]) -> None:
        self.nested_ops = nested_ops

@writer.rewrites(ops.CreateIndexOp)
def create_index_concurrently_if_not_exists(_autogen_context, _revision, op: ops.CreateIndexOp):
    op.if_not_exists = True
    op.kw["postgresql_concurrently"] = True
    return AutoCommitBlockOp([op])

would properly generated some code like this:

def upgrade() -> None:
    
    with op.get_context().autocommit_block():
        op.create_index(
            "uq_my_index",
            "my_table",
            ["col1", "col2", "col3"],
            unique=True,
            if_not_exists=True,
            postgresql_concurrently=True,
        )

Additional context
Happy to share a pull request if this feature can serve the greater good, but I could as well understand if it should not be streamlined.

Have a nice day!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the autogeneration rewrite entry point shown by writer.rewrites and the ops.MigrateOperation abstraction, then trace how generated revision code uses op.get_context().autocommit_block(). Done means nested migration operations can generate an autocommit context wrapper, including the PostgreSQL concurrent-index example, with coverage added for the generated output.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, python
Domain
databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.