MySql Deadlock in check_trigger_timeouts
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Apache Airflow version
Other Airflow 2/3 version (please specify below)
### If "Other Airflow 2/3 version" selected, which one?
2.10.5
### What happened?
Hi,
We got the exact same error as: #41429, it looks to me that we should wrap the `execute` method in `check_trigger_timeouts` so we catch the exception and can actually retry? like `adopt_or_reset_orphaned_tasks` does.
This happened for the first time shortly after we added a second triggerer instance, so this could make this issue more likely.
Thanks
Sylvain
### What you think should happen instead?
_No response_
### How to reproduce
running a scheduler and multiple triggerer.
### Operating System
Fedora 8.3
### Versions of Apache Airflow Providers
_No response_
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start by locating the check_trigger_timeouts entry point and compare its execute flow with adopt_or_reset_orphaned_tasks, especially how exceptions are handled. Reproduce the deadlock with a scheduler and multiple triggerers, then verify that the timeout check can retry after the reported MySQL error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mysql, python
- Domain
- backend, database
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100