sqlalchemy / sqlalchemy/alembic

Wrong order when renaming foreign keys

Open
#902 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

autogenerate - rendering bug mysql
Dominant language
Python
Stars
4.4k
Forks
375
PR merge metrics
No merged PRs in 30d

Description

Describe the bug
In short, I cannot rename an index on a foreign key because alembic tries to remove the index before re-creating it.

I have an existing MySQL database. It contains foreign keys which are also indices (as they should be). The indices have random names and those names are not in the code. When I run a migration, alembic generates code to rename those indices.
However, in the migration script, deletion comes before insertion, e.g.

    op.drop_index('fk_userrole_role_idx', table_name='User_Role')
    op.create_index(op.f('ix_User_Role_role_id'), 'User_Role', ['role_id'], unique=False)

and when I run the migration with alembic upgrade head, I will receive the error message

sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1553, "Cannot drop index 'fk_userrole_role_idx': needed in a foreign key constraint")

Expected behavior
I expect the instructions in the migration script to come in such an order that I get no error message, i.e., create index with new name before deleting index with old name.

To Reproduce
Excerpt from original database

DROP TABLE IF EXISTS `User_Role`;
/*!40101 SET @saved_cs_client     = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `User_Role` (
  `user_id` int NOT NULL,
  `role_id` int NOT NULL,
  KEY `fk_userId_idx` (`user_id`),
  KEY `fk_userrolw_userId_idx` (`user_id`),
  KEY `fk_userrole_role_idx` (`role_id`),
  CONSTRAINT `fk_userrole_role` FOREIGN KEY (`role_id`) REFERENCES `Role` (`id`),
  CONSTRAINT `fk_userrole_userId` FOREIGN KEY (`user_id`) REFERENCES `User` (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3;

Excerpt from code

DBUser_Role = Table(
    user_role,
    Base.metadata,
    Column("user_id", Integer, ForeignKey(user + ".id"), index=True),
    Column("role_id", Integer, ForeignKey(role + ".id"), index=True)
)

Excerpt from generated migration script

    op.drop_index('fk_userId_idx', table_name='User_Role')
    op.drop_index('fk_userrole_role_idx', table_name='User_Role')
    op.drop_index('fk_userrolw_userId_idx', table_name='User_Role')
    op.create_index(op.f('ix_User_Role_role_id'), 'User_Role', ['role_id'], unique=False)
    op.create_index(op.f('ix_User_Role_user_id'), 'User_Role', ['user_id'], unique=False)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with Alembic autogenerate's handling of the shown op.drop_index and op.create_index operations, then reproduce the failure with the User_Role schema and MySQL database described here. Compare the generated migration with the expected create-before-drop order and run alembic upgrade head; done means the migration completes without the reported OperationalError.

Written by the indexing model from the issue text.

Assessment

Tech stack
mysql, python, sqlalchemy
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.