sqlalchemy / sqlalchemy/alembic

Autogenerate renders TypeDecorator instance instead of underlying impl type

Open
#1,386 10 comments 11 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

autogenerate - rendering cookbook requested documentation typedecorator use case
Dominant language
Python
Stars
4.4k
Forks
375
PR merge metrics
No merged PRs in 30d

Description

Describe the bug

This isn't a bug per se, but a small improvement for autogenerate when using TypeDecorator.

When a TypeDecorator is used in a column definition, e.g.:

"""
File: app/models/foo.py
"""
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.types import TypeDecorator
...

class JSONBData(TypeDecorator):
   impl = JSONB

foo = Table("foo", MetaData(), Column("data", JSONBData))

The auto-generated migration ends up referencing the TypeDecorator:

op.add_column("foo", sa.Column("data", app.models.foo.JSONBData(), nullable=True))

which is annoying for two reasons:

  1. The import is not automatically rendered.
  2. The migration file has an unnecessary dependency on app, e.g. if the app/models/foo.py is refactored, we may need to update this migration file... when that could have been avoided if instead of rendering app.models.foo.JSONBData, alembic directly rendered the underlying impl: postgresql.JSONB.

I'm not sure if there are any scenarios where it is actually preferable to have the TypeDecorator in the autogenerated file. If there are use cases for it, would it be sensible to make this a config option instead of unconditional?

Expected behavior
Ideally, we'd generate the same thing as when foo = Table("foo", MetaData(), Column("data", JSONB)) is provided, i.e.:

op.add_column("foo", sa.Column("data", postgresql.JSONB(astext_type=Text()), nullable=True))

To Reproduce

Test case:

diff --git a/tests/test_autogen_render.py b/tests/test_autogen_render.py
index eeeb92e..9755869 100644
--- a/tests/test_autogen_render.py
+++ b/tests/test_autogen_render.py
@@ -33,6 +33,7 @@ from sqlalchemy.sql import false
 from sqlalchemy.sql import literal_column
 from sqlalchemy.sql import table
 from sqlalchemy.types import TIMESTAMP
+from sqlalchemy.types import TypeDecorator
 from sqlalchemy.types import UserDefinedType
 
 from alembic import autogenerate
@@ -1078,6 +1079,21 @@ class AutogenRenderTest(TestBase):
             "server_default='5', nullable=True))",
         )
 
+    def test_render_add_column_type_decorator(self):
+        self.autogen_context.opts["user_module_prefix"] = None
+
+        class MyType(TypeDecorator):
+            impl = Integer
+
+        op_obj = ops.AddColumnOp(
+            "foo", Column("x", MyType, server_default="5")
+        )
+        eq_ignore_whitespace(
+            autogenerate.render_op_text(self.autogen_context, op_obj),
+            "op.add_column('foo', sa.Column('x', sa.Integer(), "
+            "server_default='5', nullable=True))",
+        )
+
     @testing.emits_warning("Can't validate argument ")
     def test_render_add_column_custom_kwarg(self):
         col = Column(

Error

When running tox -e py-sqlalchemy -- tests/test_autogen_render.py with the above patch:

  File "/Users/saif/contrib/alembic/tests/test_autogen_render.py", line 1091, in test_render_add_column_type_decorator
    eq_ignore_whitespace(
  File "/Users/saif/contrib/alembic/alembic/testing/assertions.py", line 111, in eq_ignore_whitespace
    assert a == b, msg or "%r != %r" % (a, b)
           ^^^^^^
AssertionError: "op.add_column('foo', sa.Column('x', tests.test_autogen_render.MyType(), server_default='5', nullable=True))" != "op.add_column('foo', sa.Column('x', sa.Integer(), server_default='5', nullable=True))"

Versions.

  • OS: macOS 14.1.2
  • Python: 3.11.6
  • Alembic: 1.13.1
  • SQLAlchemy: 1.3.24 / 2.0.23
  • Database: Postgres
  • DBAPI: psycopg2

Have a nice day!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with tests/test_autogen_render.py and run tox -e py-sqlalchemy -- tests/test_autogen_render.py, focusing on AutogenRenderTest and autogenerate.render_op_text. Trace how AddColumnOp renders TypeDecorator instances, then verify the test produces the underlying implementation type without an application-model dependency.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, python
Domain
database, tooling
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
66/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.