Improve DB migration tests in CI
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 484
Description
### Body
Our DB migration tests in CI run on empty tables and some of the migrations perform some data manipulation during down/up migration. By having empty tables we are skipping over this logic.
We should likely add some steps before and in-between the migration to populate the database witth data - likely by running a few dags maybe runniing a few other commands that modify the data (connections/variables etc.).
Also, it seems that some of the migrations might behave differently depending on what is the "to" migration - for example in case of 2.9.0 -> 2.8.4 migration, the original migration in #38743 failed, while 2.0.0 did not and we have to either change our CI testing or (I'd say more likely) make it a bit more "Target independent"
### Committer
- [X] I acknowledge that I am a maintainer/committer of the Apache Airflow project.
Contributor guide
Research direction
Start by reviewing the CI database migration test setup and the failing comparison described in #38743. Identify how migration paths currently use empty tables and which upgrade/downgrade targets are covered; done means the tests exercise populated data and behave consistently across target migrations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ci-cd, database, testing
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100