apache / apache/airflow

Add an option to run backfill while keeping dag execution suspended

Open
#60,768 2 comments 0 reactions 0 assignees View on GitHub
area:backfill kind:feature needs-triage
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Description

As a maintainer
I would like to run backfill while suspending dag execution
In order to backfill without executing dag every day
In order to avoid useless computation and log

### Use case/motivation

To perform a backfill, a dag must be triggered by cron and also it must be activated. In our case, we have a dag to reprocess old data, but we cannot run it dayly via cron because the data is not available.

Image

So we want to run only backfill while keeping daily dag execution suspended to avoid useless computation.

Actually, we have an option `Unpause XXXX on trigger`. We could add a new option `Run backfill and keep dag suspended` to run only backfill

### Related issues

_No response_

### Are you willing to submit a PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.