apache / apache/airflow

MySql Deadlock in check_trigger_timeouts

Open
#56,670 0 comments 0 reactions 0 assignees View on GitHub
_eol_affected_version:2.10 area:core area:Triggerer kind:bug needs-triage
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Apache Airflow version

Other Airflow 2/3 version (please specify below)

### If "Other Airflow 2/3 version" selected, which one?

2.10.5

### What happened?

Hi,

We got the exact same error as: #41429, it looks to me that we should wrap the `execute` method in `check_trigger_timeouts` so we catch the exception and can actually retry? like `adopt_or_reset_orphaned_tasks` does.

This happened for the first time shortly after we added a second triggerer instance, so this could make this issue more likely.

Thanks
Sylvain

### What you think should happen instead?

_No response_

### How to reproduce

running a scheduler and multiple triggerer.

### Operating System

Fedora 8.3

### Versions of Apache Airflow Providers

_No response_

### Deployment

Official Apache Airflow Helm Chart

### Deployment details

_No response_

### Anything else?

_No response_

### Are you willing to submit PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start by locating the check_trigger_timeouts entry point and compare its execute flow with adopt_or_reset_orphaned_tasks, especially how exceptions are handled. Reproduce the deadlock with a scheduler and multiple triggerers, then verify that the timeout check can retry after the reported MySQL error.

Written by the indexing model from the issue text.

Assessment

Tech stack
mysql, python
Domain
backend, database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.