sqlalchemy / sqlalchemy/alembic
Alembic always detects changes on indexes that use to_tsvector()
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.4k
- Forks
- 375
- PR merge metrics
- No merged PRs in 30d
Description
Since I upgraded to Alembic 1.13.1, SQLAlchemy 2.0.25, Flask-SQLAlchemy 3.1.1, Flask 3.0.0 and Python 3.12, everytime I run flask db migrate Alembic detects unexisting changes on indexes that use to_tsvector() for full text search using PostgreSQL 16. It keeps creating migrations to drop and recreate those unchanged indexes again and again.
The Alembic output is:
INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
INFO [alembic.ddl.postgresql] Detected sequence named 'post_reading_id_seq' as owned by integer column 'post_reading(id)', assuming SERIAL and omitting
INFO [alembic.ddl.postgresql] Detected sequence named 'chapter_comment_reaction_id_seq' as owned by integer column 'chapter_comment_reaction(id)', assuming SERIAL and omitting
INFO [alembic.ddl.postgresql] Detected sequence named 'page_id_seq' as owned by integer column 'page(id)', assuming SERIAL and omitting
INFO [alembic.ddl.postgresql] Detected sequence named 'book_reservation_id_seq' as owned by integer column 'book_reservation(id)', assuming SERIAL and omitting
INFO [alembic.ddl.postgresql] Detected sequence named 'category_search_id_seq' as owned by integer column 'category_search(id)', assuming SERIAL and omitting
INFO [alembic.ddl.postgresql] Detected sequence named 'chapter_reading_id_seq' as owned by integer column 'chapter_reading(id)', assuming SERIAL and omitting
INFO [alembic.ddl.postgresql] Detected sequence named 'post_comment_reaction_id_seq' as owned by integer column 'post_comment_reaction(id)', assuming SERIAL and omitting
INFO [alembic.ddl.postgresql] Detected sequence named 'alert_id_seq' as owned by integer column 'alert(id)', assuming SERIAL and omitting
INFO [alembic.ddl.postgresql] Detected sequence named 'contact_message_id_seq' as owned by integer column 'contact_message(id)', assuming SERIAL and omitting
INFO [alembic.ddl.postgresql] Detected sequence named 'chapter_reaction_id_seq' as owned by integer column 'chapter_reaction(id)', assuming SERIAL and omitting
INFO [alembic.ddl.postgresql] Detected sequence named 'broadcast_message_id_seq' as owned by integer column 'broadcast_message(id)', assuming SERIAL and omitting
INFO [alembic.ddl.postgresql] Detected sequence named 'snippet_id_seq' as owned by integer column 'snippet(id)', assuming SERIAL and omitting
INFO [alembic.ddl.postgresql] Detected sequence named 'text_search_id_seq' as owned by integer column 'text_search(id)', assuming SERIAL and omitting
INFO [alembic.ddl.postgresql] Detected sequence named 'post_reaction_id_seq' as owned by integer column 'post_reaction(id)', assuming SERIAL and omitting
INFO [alembic.autogenerate.compare] Detected changed index 'ix_category_ts' on 'category': expression #1 "to_tsvector('spanish'::regconfig, (name::text || ' '::text) || hashtag::text)" to "to_tsvector('spanish', name || ' ' || hashtag)"
INFO [alembic.autogenerate.compare] Detected changed index 'ix_chapter_ts' on 'chapter': expression #1 "to_tsvector('spanish'::regconfig, (((title::text || ' '::text) || teaser::text) || ' '::text) || raw_text)" to "to_tsvector('spanish', chapter.title || ' ' || teaser || ' ' || raw_text)"
INFO [alembic.autogenerate.compare] Detected changed index 'ix_post_ts' on 'post': expression #1 "to_tsvector('spanish'::regconfig, (((title::text || ' '::text) || teaser::text) || ' '::text) || raw_text)" to "to_tsvector('spanish', post.title || ' ' || teaser || ' ' || raw_text)"
Generating /app/migrations/versions/673f9d0115e2_.py ... done
The affected models look as follows (simplified):
class Chapter(BasePublication):
part_id = db.Column(db.Integer, db.ForeignKey('part.id', ondelete='SET NULL'), index=True)
classification = db.relationship('Part', backref=db.backref('publications', lazy='dynamic'))
categories = db.relationship('Category', secondary='chapter_category',
backref=db.backref('chapters', lazy='dynamic'), lazy='dynamic')
__ts_vector__ = sa.func.to_tsvector(
sa.literal('spanish'), sa.text("chapter.title || ' ' || teaser || ' ' || raw_text"))
__table_args__ = (
db.Index('ix_chapter_ts', __ts_vector__, postgresql_using='gin'),
db.UniqueConstraint('number', 'fragment', name='unique_chapter_number_fragment'),
)
class Post(BasePublication):
topic_id = db.Column(db.Integer, db.ForeignKey('topic.id', ondelete='SET NULL'), index=True)
classification = db.relationship('Topic', backref=db.backref('publications', lazy='dynamic'))
categories = db.relationship(
'Category', secondary='post_category', backref=db.backref('posts', lazy='dynamic'), lazy='dynamic')
__ts_vector__ = sa.func.to_tsvector(
sa.literal('spanish'), sa.text("post.title || ' ' || teaser || ' ' || raw_text"))
__table_args__ = (
db.Index('ix_post_ts', __ts_vector__, postgresql_using='gin'),
db.UniqueConstraint('number', 'fragment', name='unique_post_number_fragment'),
)
class Category(BaseModel):
name = db.Column(db.String(50), nullable=False, unique=True)
hashtag = db.Column(db.String(51), nullable=False, unique=True)
__ts_vector__ = sa.func.to_tsvector(
sa.literal('spanish'), sa.text("name || ' ' || hashtag"))
__table_args__ = (
db.Index('ix_category_ts', __ts_vector__, postgresql_using='gin'),
)
One of the multiple generated migrations looks like follows:
def upgrade():
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table('category', schema=None) as batch_op:
batch_op.drop_index('ix_category_ts', postgresql_using='gin')
batch_op.create_index('ix_category_ts', [sa.text("to_tsvector('spanish', name || ' ' || hashtag)")], unique=False, postgresql_using='gin')
with op.batch_alter_table('chapter', schema=None) as batch_op:
batch_op.drop_index('ix_chapter_ts', postgresql_using='gin')
batch_op.create_index('ix_chapter_ts', [sa.text("to_tsvector('spanish', chapter.title || ' ' || teaser || ' ' || raw_text)")], unique=False, postgresql_using='gin')
with op.batch_alter_table('post', schema=None) as batch_op:
batch_op.drop_index('ix_post_ts', postgresql_using='gin')
batch_op.create_index('ix_post_ts', [sa.text("to_tsvector('spanish', post.title || ' ' || teaser || ' ' || raw_text)")], unique=False, postgresql_using='gin')
# ### end Alembic commands ###
def downgrade():
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table('post', schema=None) as batch_op:
batch_op.drop_index('ix_post_ts', postgresql_using='gin')
batch_op.create_index('ix_post_ts', [sa.text("to_tsvector('spanish'::regconfig, (((title::text || ' '::text) || teaser::text) || ' '::text) || raw_text)")], unique=False, postgresql_using='gin')
with op.batch_alter_table('chapter', schema=None) as batch_op:
batch_op.drop_index('ix_chapter_ts', postgresql_using='gin')
batch_op.create_index('ix_chapter_ts', [sa.text("to_tsvector('spanish'::regconfig, (((title::text || ' '::text) || teaser::text) || ' '::text) || raw_text)")], unique=False, postgresql_using='gin')
with op.batch_alter_table('category', schema=None) as batch_op:
batch_op.drop_index('ix_category_ts', postgresql_using='gin')
batch_op.create_index('ix_category_ts', [sa.text("to_tsvector('spanish'::regconfig, (name::text || ' '::text) || hashtag::text)")], unique=False, postgresql_using='gin')
# ### end Alembic commands ###
When I was using Alembic 1.7.6, SQLAlchemy 1.4.31, Flask-SQLAlchemy 2.5.1, Flask 2.0.3 and Python 3.10 this didn't happen as there was no support for this kind of indexes at that moment.
I hope this can get resolved soon.
Best regards.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the issue with flask db migrate using the shown SQLAlchemy models and PostgreSQL full-text indexes. Compare the generated migration at /app/migrations/versions/673f9d0115e2_.py with the reported Alembic autogenerate output. Done means unchanged to_tsvector() indexes no longer produce repeated drop-and-recreate migrations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, python, sqlalchemy
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100