apache / apache/airflow

Dynamically change task retries

Open
#64,483 1 comment 0 reactions 0 assignees View on GitHub
area:core kind:feature needs-triage
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Description

The ability for tasks to update their retry count at execution time. The airflow sdk does not currently have a way to mutate task retry metadata so you are stuck with the retry count defined at init.

### Use case/motivation

I have KPO tasks that dynamically adjust resource usage in response to OOM events. I would love the ability to add retries after OOM events but not after other failures.

I can also see this being useful in case of api request timeout code in task execution.

### Related issues

_No response_

### Are you willing to submit a PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.