Getting Duplicate entry issue in dag_tag
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 484
Description
### Apache Airflow version
Other Airflow 2 version (please specify below)
### If "Other Airflow 2 version" selected, which one?
2.5.0
### What happened?
When updating the tags for a dag for upper-case to lower-case the scheduler give this info and the Dag does not appear on the UI.
`sqlalchemy.exc.PendingRollbackError: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (MySQLdb._exceptions.IntegrityError) (1062, "Duplicate entry 'pacman-Campaign-711_PRD-1671' for key 'dag_tag.PRIMARY'")
[parameters: (('pacman', 'Campaign-711_PRD-1671'), ('pu', 'Campaign-711_PRD-1671'))] `
### What you think should happen instead?
There should be no such error if the Dag tag is updated
### How to reproduce
Im using MYSQL Server version: 8.0.28 Source distribution
``` CREATE TABLE `dag_tag` (
`name` varchar(100) NOT NULL,
`dag_id` varchar(250) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
PRIMARY KEY (`name`,`dag_id`),
KEY `dag_tag_dag_id_fkey` (`dag_id`),
CONSTRAINT `dag_tag_dag_id_fkey` FOREIGN KEY (`dag_id`) REFERENCES `dag` (`dag_id`) ON DELETE CASCADE ON UPDATE RESTRICT
) ENGINE=InnoDB DEFAULT CHARSET=latin1```
Update the tags of a dag from upper-case to lower-case.
### Operating System
centos 7
### Versions of Apache Airflow Providers
_No response_
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start by tracing the scheduler's DAG-tag update path and the dag_tag primary-key behavior when a tag changes case. Reproduce the update against MySQL 8.0.28 using the schema and steps in the issue, then verify that the transaction completes without a duplicate-entry error and that the DAG remains visible in the UI.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mysql, python, sqlalchemy
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100