sqlalchemy / sqlalchemy/alembic

postgresql serial column detection might fail

Open
#1,565 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug postgresql
Dominant language
Python
Stars
4.4k
Forks
375
PR merge metrics
No merged PRs in 30d

Description

Describe the bug
There is a probability that serial column was not detected.
This can happen when the table is referenced by another table.

Expected behavior
The result of autogererate should't contain server default for column id.

To Reproduce

import sqlalchemy as sa
from alembic.autogenerate.api import produce_migrations
from alembic.autogenerate.api import render_python_code
from alembic.migration import MigrationContext


def filter_by_schema(schema_name: str):
    def _filter_by_schema(name, type_, parent_names):
        if type_ == "schema":
            return name == schema_name
        else:
            return True

    return _filter_by_schema


engine = sa.create_engine("postgresql://name:pass@host:1234/tmp")
with engine.connect() as conn, conn.begin():
    conn.execute(
        sa.text(
            """
        DROP SCHEMA IF EXISTS test CASCADE;
        CREATE SCHEMA test;
        CREATE TABLE test.account1(id SERIAL PRIMARY KEY);
        CREATE TABLE test.account2(id SERIAL PRIMARY KEY, id2 int REFERENCES test.account1(id));
        """
        )
    )
metadata = sa.MetaData(schema="test")
mc = MigrationContext.configure(
    connection=engine.connect(),
    opts={"include_name": filter_by_schema("test"), "include_schemas": True},
)
migration_script = produce_migrations(mc, metadata)
downgrade_code = render_python_code(migration_script.downgrade_ops)
print(downgrade_code)

Error
It will produce two outputs and the first one is incorrect.

# ### commands auto generated by Alembic - please adjust! ###
    op.create_table('account1',
    sa.Column('id', sa.INTEGER(), server_default=sa.text("nextval('test.account1_id_seq'::regclass)"), autoincrement=True, nullable=False),
    sa.PrimaryKeyConstraint('id', name='account1_pkey'),
    schema='test',
    postgresql_ignore_search_path=False
    )
    op.create_table('account2',
    sa.Column('id', sa.INTEGER(), autoincrement=True, nullable=False),
    sa.Column('id2', sa.INTEGER(), autoincrement=False, nullable=True),
    sa.ForeignKeyConstraint(['id2'], ['test.account1.id'], name='account2_id2_fkey'),
    sa.PrimaryKeyConstraint('id', name='account2_pkey'),
    schema='test'
    )
    # ### end Alembic commands ###
# ### commands auto generated by Alembic - please adjust! ###
    op.create_table('account2',
    sa.Column('id', sa.INTEGER(), autoincrement=True, nullable=False),
    sa.Column('id2', sa.INTEGER(), autoincrement=False, nullable=True),
    sa.ForeignKeyConstraint(['id2'], ['test.account1.id'], name='account2_id2_fkey'),
    sa.PrimaryKeyConstraint('id', name='account2_pkey'),
    schema='test'
    )
    op.create_table('account1',
    sa.Column('id', sa.INTEGER(), autoincrement=True, nullable=False),
    sa.PrimaryKeyConstraint('id', name='account1_pkey'),
    schema='test'
    )
    # ### end Alembic commands ###

Versions.

  • OS: macos 14.6.1
  • Python: 3.10.9
  • Alembic: 1.13.3
  • SQLAlchemy: 2.0.36
  • Database: PostgreSQL 15.7
  • DBAPI: psycopg2-binary 2.9.10

Additional context
If table is not fererenced by other table, the result is always correct.
If I make both tables reference each other, the first table in autogenerate is alway incorrect.

ALTER TABLE test.account1 ADD FOREIGN KEY (id) REFERENCES test.account2(id);

I guess the listen hook migth fail with foreign keys, not sure since havn't dig into code much.

Have a nice day!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the case with PostgreSQL using produce_migrations and render_python_code from alembic.autogenerate.api, plus MigrationContext and the two referenced tables. Start by comparing the generated operations when account1 is referenced by account2. Done means autogenerate does not add an incorrect server_default for account1.id and produces consistent output for the reported schema.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, python, sqlalchemy
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.