apache / apache/airflow

Duplicate indices on MySQL

Open
#53,509 4 comments 0 reactions 0 assignees View on GitHub
area:core area:MetaDB kind:bug
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Apache Airflow version

3.0.3

### If "Other Airflow 2 version" selected, which one?

_No response_

### What happened?

We are checking the indices that Airflow creates on different tables to find any performance issues and we noticed on `MySQL`, there are duplicate indices created on `dag_run` table:
- KEY `idx_dag_run_running_dags` (`state`,`dag_id`)
- KEY `idx_dag_run_queued_dags` (`state`,`dag_id`)

Even though index/key names are different, since the columns and their order are same, it is essentially the duplicate. And this will effect any `INSERT` operations and potentially `SELECT` statements too, since `MySQL` will be confused about which index among the duplicates should be used.

### What you think should happen instead?

_No response_

### How to reproduce

N/A

### Operating System

MySQL

### Versions of Apache Airflow Providers

_No response_

### Deployment

Other Docker-based deployment

### Deployment details

MySQL v8.0

### Anything else?

_No response_

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start by locating the schema or migration definitions for the dag_run table and inspect how the MySQL indexes idx_dag_run_running_dags and idx_dag_run_queued_dags are created. Verify whether both indexes are generated with the same column order, then determine the expected single-index schema and add or update coverage for the migration if the repository provides it.

Written by the indexing model from the issue text.

Assessment

Tech stack
mysql, python
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.