apache / apache/airflow

Improve DB migration tests in CI

Open
#38,744 1 comment 0 reactions 0 assignees View on GitHub
kind:meta
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 7h
Merged PRs (30d)
484

Description

### Body

Our DB migration tests in CI run on empty tables and some of the migrations perform some data manipulation during down/up migration. By having empty tables we are skipping over this logic.

We should likely add some steps before and in-between the migration to populate the database witth data - likely by running a few dags maybe runniing a few other commands that modify the data (connections/variables etc.).

Also, it seems that some of the migrations might behave differently depending on what is the "to" migration - for example in case of 2.9.0 -> 2.8.4 migration, the original migration in #38743 failed, while 2.0.0 did not and we have to either change our CI testing or (I'd say more likely) make it a bit more "Target independent"

### Committer

- [X] I acknowledge that I am a maintainer/committer of the Apache Airflow project.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the CI database migration test setup and the failing comparison described in #38743. Identify how migration paths currently use empty tables and which upgrade/downgrade targets are covered; done means the tests exercise populated data and behave consistently across target migrations.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ci-cd, database, testing
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.