Block migrations on MySQL when using bad collations/encoding
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Apache Airflow version
main (development)
### Operating System
MacOS 11.6
### Versions of Apache Airflow Providers
_No response_
### Deployment
Virtualenv installation
### Deployment details
_No response_
### What happened
Currently we happily attempt to migrate the database regardless of what collation and encoding is used.
This can cause migrations to fail unexpectedly, and with MySQL's lack of DDL transactions can leave the database
in a half-migrated broken state (see #18939 and https://github.com/apache/airflow/issues/18939#issuecomment-953395944).
### What you expected to happen
We should identify when we have bad values for encoding/collation, block migrations from happening, and offer detailed instructions on how to fix it.
### How to reproduce
Create a MySQL database without specifying the encoding/collation as documented in [Setting up a MySQL database](https://airflow.apache.org/docs/apache-airflow/stable/howto/set-up-database.html#setting-up-a-mysql-database).
### Anything else
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start by reproducing the problem using the MySQL setup instructions in the “Setting up a MySQL database” documentation, with an unspecified encoding or collation. Trace the migration entry point and determine where invalid values can be detected before migrations run. Done means invalid MySQL settings block migration and provide detailed correction instructions without leaving a half-migrated database.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mysql, python
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100