apache / apache/airflow

allow configuring dataset model uri collation

Open
#31,373 5 comments 0 reactions 0 assignees View on GitHub
area:MetaDB kind:feature
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 7h
Merged PRs (30d)
484

Description

### Description

Currently, the dataset model's uri column has a hard code collation. The 'latin1_general_cs' collation is not supported by some MySQL compatible database, like TiDB. So it's not possible to use the database with Airflow while most other features are compatible.

https://github.com/apache/airflow/blob/17e5e7b3b0ceae9f7e25bbee2d4beb62a9aaab39/airflow/models/dataset.py#L59

### Use case/motivation

Allow users to customize the collation while still default to 'latin1_general_cs'.

### Related issues

_No response_

### Are you willing to submit a PR?

- [X] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start in airflow/models/dataset.py at the referenced uri column and inspect how its hard-coded collation is declared. Trace nearby model tests or database-schema test coverage, then verify that users can override the collation while latin1_general_cs remains the default and compatible databases are covered.

Written by the indexing model from the issue text.

Assessment

Tech stack
mysql, python
Domain
databases
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.