apache / apache/airflow

Getting Duplicate entry issue in dag_tag

Closed
#50,815 9 comments 0 reactions 0 assignees View on GitHub
_eol_affected_version:2.5 area:core area:MetaDB kind:bug Stale Bug Report
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 7h
Merged PRs (30d)
484

Description

### Apache Airflow version

Other Airflow 2 version (please specify below)

### If "Other Airflow 2 version" selected, which one?

2.5.0

### What happened?

When updating the tags for a dag for upper-case to lower-case the scheduler give this info and the Dag does not appear on the UI.

`sqlalchemy.exc.PendingRollbackError: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (MySQLdb._exceptions.IntegrityError) (1062, "Duplicate entry 'pacman-Campaign-711_PRD-1671' for key 'dag_tag.PRIMARY'")
[parameters: (('pacman', 'Campaign-711_PRD-1671'), ('pu', 'Campaign-711_PRD-1671'))] `

### What you think should happen instead?

There should be no such error if the Dag tag is updated

### How to reproduce

Im using MYSQL Server version: 8.0.28 Source distribution

``` CREATE TABLE `dag_tag` (
`name` varchar(100) NOT NULL,
`dag_id` varchar(250) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
PRIMARY KEY (`name`,`dag_id`),
KEY `dag_tag_dag_id_fkey` (`dag_id`),
CONSTRAINT `dag_tag_dag_id_fkey` FOREIGN KEY (`dag_id`) REFERENCES `dag` (`dag_id`) ON DELETE CASCADE ON UPDATE RESTRICT
) ENGINE=InnoDB DEFAULT CHARSET=latin1```

Update the tags of a dag from upper-case to lower-case.

### Operating System

centos 7

### Versions of Apache Airflow Providers

_No response_

### Deployment

Official Apache Airflow Helm Chart

### Deployment details

_No response_

### Anything else?

_No response_

### Are you willing to submit PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start by tracing the scheduler's DAG-tag update path and the dag_tag primary-key behavior when a tag changes case. Reproduce the update against MySQL 8.0.28 using the schema and steps in the issue, then verify that the transaction completes without a duplicate-entry error and that the DAG remains visible in the UI.

Written by the indexing model from the issue text.

Assessment

Tech stack
mysql, python, sqlalchemy
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.