tortoise / tortoise/tortoise-orm
DROP INDEX in migrations is not schema-qualified (PostgreSQL/Oracle) — breaks RemoveIndex/RemoveConstraint for models outside the default schema
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 5.6k
- Forks
- 516
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 9
Description
Summary
For models placed in a non-default PostgreSQL schema (Meta.schema = "..."), running a migration that drops an index (ops.RemoveIndex) or a partial unique constraint (ops.RemoveConstraint) fails with index "..." does not exist, even though the index/constraint is present in the database — it simply isn't visible under the connection's default search_path.
Tested against tortoise-orm==1.1.7, and confirmed unchanged against the current 1.1.8 release tag as well as the develop branch HEAD (checked 2026-09-19).
Root cause / affected locations
-
tortoise/migrations/schema_editor/base.py:51DROP_INDEX_TEMPLATE = 'DROP INDEX "{name}"'remove_index()(same file, lines 816–823) does compute a schema-qualified value and pass it in:async def remove_index(self, model: type[Model], index: Index) -> None: index_name = self._index_name_for_model(model, index) await self._run_sql( self.DROP_INDEX_TEMPLATE.format( name=index_name, table=self._qualify_table_name(model._meta.db_table, model._meta.schema), ) )but since the template has no
{table}placeholder,str.format()silently discards the extra keyword argument. The emitted SQL is always the unqualifiedDROP INDEX "index_name". -
tortoise/migrations/schema_editor/base_postgres.py:130-142(remove_constraint, used for partialUniqueConstraints):constraint_name = self._constraint_name_for_model(model, resolved_constraint) await self._run_sql(self.DROP_INDEX_TEMPLATE.format(name=constraint_name))Same template, and here no table/schema value is passed at all.
For comparison, the creation path is correctly schema-qualified — INDEX_CREATE_TEMPLATE = 'CREATE INDEX "{index_name}" ON {table_name} ({fields}){extra};' uses {table_name} via _qualify_table_name(), so the index is physically created inside the target schema (Postgres always creates an index in its table's schema). It is only the drop path that loses the qualification, which produces the specific and slightly confusing symptom of "index does not exist" for an index that provably does exist.
Affected backends
- PostgreSQL — affected, reproduced live below (
AsyncpgSchemaEditor/PsycopgSchemaEditor, both inheritbase_postgres.BasePostgresSchemaEditorwithout overridingDROP_INDEX_TEMPLATE). - Oracle (
OracleSchemaEditor) — inherits the same unmodifiedDROP_INDEX_TEMPLATEfrombase.pyand uses the same ANSI/schema-qualification mixin as Postgres. Very likely affected by the same mechanism, but not independently verified against a live Oracle instance — flagging this from static code reading only. - SQLite (
SqliteSchemaEditor) — redeclares the identical unqualified template, but is not practically affected in typical use:SqliteQuotingMixin._qualify_table_name()ignoresschemaentirely ("SQLite does not support database schemas in the standard sense"), so there is normally nothing to qualify. - MySQL / MSSQL — not affected. Both override
DROP_INDEX_TEMPLATEwith their own table-scoped syntax (DROP INDEX `{name}` ON {table}/DROP INDEX [{name}] ON {table}), which matches how those dialects actually scope index names to a table.
Expected vs. actual behavior
-
Expected:
ops.RemoveIndex/ops.RemoveConstraintdrop the index/constraint regardless of which schema the model's table lives in. -
Actual: on PostgreSQL, this only works when the target schema happens to be on the connection's
search_path(default"$user", public). For any model in an explicitly named, non-default schema, the migration aborts with e.g.:asyncpg.exceptions.UndefinedObjectError: index "idx_..." does not exist
Minimal reproduction
Self-contained script, only depends on tortoise-orm and asyncpg against any reachable PostgreSQL instance. It creates its own throwaway schema and drops it again at the end.
"""Minimal repro: tortoise-orm DROP INDEX is not schema-qualified on PostgreSQL."""
import asyncio
import os
import traceback
import asyncpg
from tortoise import Tortoise, fields
from tortoise.indexes import Index
from tortoise.migrations.schema_editor.asyncpg import AsyncpgSchemaEditor
from tortoise.models import Model
SCHEMA = "repro_schema"
INDEX_NAME = "idx_repro_value"
class ReproModel(Model):
id = fields.IntField(primary_key=True)
value = fields.IntField()
class Meta:
schema = SCHEMA
indexes = [Index(fields=("value",), name=INDEX_NAME)]
async def admin_exec(host, port, user, password, db, sql):
conn = await asyncpg.connect(host=host, port=port, user=user, password=password, database=db)
try:
await conn.execute(sql)
finally:
await conn.close()
async def main():
host = os.environ["POSTGRES_HOST"]
port = int(os.environ.get("POSTGRES_PORT", "5432"))
user = os.environ["POSTGRES_USER"]
password = os.environ["POSTGRES_PASSWORD"]
db = os.environ["POSTGRES_DB"]
await admin_exec(host, port, user, password, db, f'DROP SCHEMA IF EXISTS "{SCHEMA}" CASCADE')
await admin_exec(host, port, user, password, db, f'CREATE SCHEMA "{SCHEMA}"')
try:
await Tortoise.init(
db_url=f"asyncpg://{user}:{password}@{host}:{port}/{db}",
modules={"models": ["__main__"]},
)
conn = Tortoise.get_connection("default")
await Tortoise.generate_schemas(safe=True)
rows = await conn.execute_query(
"SELECT schemaname, indexname FROM pg_indexes WHERE indexname = $1", [INDEX_NAME]
)
print(f"index physically lives in: {rows[1]}") # -> schema 'repro_schema', as expected
editor = AsyncpgSchemaEditor(conn, atomic=False)
index_obj = ReproModel._meta.indexes[0]
try:
await editor.remove_index(ReproModel, index_obj)
print("UNEXPECTED: remove_index() succeeded")
except Exception as exc:
print(f"REPRODUCED: {type(exc).__name__}: {exc}")
traceback.print_exc()
await Tortoise.close_connections()
finally:
await admin_exec(host, port, user, password, db, f'DROP SCHEMA IF EXISTS "{SCHEMA}" CASCADE')
if __name__ == "__main__":
asyncio.run(main())
Output against a real PostgreSQL instance:
index physically lives in: <Record schemaname='repro_schema' indexname='idx_repro_value'>
REPRODUCED: OperationalError: index "idx_repro_value" does not exist
Real-world impact
This is not just a synthetic edge case — we hit it repeatedly in a production codebase that places its application tables in a non-default PostgreSQL schema. Across several migrations over multiple months we ended up converging on the same local workaround: wrapping the affected ops.RemoveIndex/ops.RemoveConstraint operation in SET LOCAL search_path TO <our_schema>, public; / SET LOCAL search_path TO DEFAULT;. That works, but it's easy to miss (our own test suite didn't catch it either, since tests build the schema fresh from the models and never exercise the migration executor's RemoveIndex path against an existing database).
Suggested fix direction
Rather than mirroring MySQL/MSSQL's DROP INDEX name ON table form (which isn't valid PostgreSQL/Oracle syntax — neither dialect supports an ON table clause on DROP INDEX), the index/constraint name itself should be schema-qualified, e.g. by reusing the existing _qualify_table_name() helper (it just qualifies an identifier with a schema prefix, despite the name):
async def remove_index(self, model: type[Model], index: Index) -> None:
index_name = self._index_name_for_model(model, index)
qualified_name = self._qualify_table_name(index_name, model._meta.schema)
await self._run_sql(f"DROP INDEX {qualified_name}")
and analogously in BasePostgresSchemaEditor.remove_constraint() for constraint_name. This would apply equally to Oracle, which shares the same quoting mixin.
We're not in a position to contribute a PR for this (constraints on our side), but happy to provide more detail or verify a proposed fix against our workaround cases if useful.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in tortoise/migrations/schema_editor/base.py at DROP_INDEX_TEMPLATE and remove_index(), then inspect base_postgres.py at remove_constraint(). Run the supplied PostgreSQL reproduction against a model in a non-default schema, and verify that RemoveIndex and RemoveConstraint can drop the existing index or constraint without changing search_path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, python
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 70/100