sqlalchemy / sqlalchemy/alembic

re-introduce hardcoded per-dialect type argument comparison rules for specific cases such as PG VARCHAR length missing or not

Open
#749 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

autogenerate - detection data types postgresql typedecorator
Dominant language
Python
Stars
4.4k
Forks
375
PR merge metrics
No merged PRs in 30d

Description

The bug
Prior to alembic version 1.4.0, adding a length to a String type would be auto-detected and would result in a migration that included the change. Starting with version 1.4.0, this change is not included in a migration.

Expected behavior
For a given column, adding length to types.String should result in a migration that includes the change.

To Reproduce

  1. Install alembic < 1.4.0 (e.g. 1.3.3) and create an initial migration using autodetect. I used the following model:
from sqlalchemy import Column, Integer, String
from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()


class User(Base):
    __tablename__ = 'user'

    id = Column(Integer, primary_key=True)
    name = Column(types.String)
  1. Run alembic revision --autogenerate -m "initial" and observe that it produces something like the following migration:
from alembic import op
import sqlalchemy as sa


revision = 'dd4a6ee1bfd2'
down_revision = None
branch_labels = None
depends_on = None


def upgrade():
    op.create_table('user',
    sa.Column('id', sa.Integer(), nullable=False),
    sa.Column('name', sa.String(), nullable=True),
    sa.PrimaryKeyConstraint('id')
    )


def downgrade():
    op.drop_table('user')
  1. Run alembic upgrade head, update types.String to types.String(10), and generate another migration with autogenerate. Observe something like the following migration:
from alembic import op
import sqlalchemy as sa


revision = 'cca9e3305312'
down_revision = 'dd4a6ee1bfd2'
branch_labels = None
depends_on = None


def upgrade():
    op.alter_column('user', 'name',
               existing_type=sa.VARCHAR(),
               type_=sa.String(length=10),
               existing_nullable=True)


def downgrade():
    op.alter_column('user', 'name',
               existing_type=sa.String(length=10),
               type_=sa.VARCHAR(),
               existing_nullable=True)

Repeat these steps for alembic >= 1.4.0 and you'll notice that the migration produced in step 3 is empty (there are no operations).

Note that if the String were to start out with a defined length, changing that length would result in a migration with the change for alembic >= 1.4.0.

Versions.

  • OS: macOS Catalina (10.15.6)
  • Python: 3.8
  • Alembic: >= 1.4.0
  • SQLAlchemy: 1.3.20
  • Database: postgres

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with PostgreSQL, SQLAlchemy 1.3.20, and Alembic 1.4 or later using the two autogenerate steps shown. Start by tracing the autogenerate type-comparison path for String and dialect-specific arguments. Done means adding a length to an initially unbounded String produces an alter-column migration while preserving existing length-change behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, python, sqlalchemy
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
54/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.