apache / apache/airflow

Removing DAG runs for a specific time range

Open
#52,882 0 comments 1 reaction 0 assignees View on GitHub
area:CLI kind:feature needs-triage
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 7h
Merged PRs (30d)
484

Description

### Description

Allow removing DAG runs from CLI or/and UI, additionally allowing to limit time range by specifying _before_ and/or period for logical dates. Eg. `airflow dags delete-runs --before 2025-01-01 00:00:00+01:00 --after 2024-01-01 00:00:00+01:00` or similar form in UI.

### Use case/motivation

I'm aware that in Airflow 3.0, individual DAG runs can be removed via the API. However, deleting a large number of DAG runs can be tedious and typically requires additional scripting to do efficiently. Having a built-in feature for this would be a helpful workaround for the following issues:

**Updating executor_config for historical runs**
In some cases, we need to rerun DAG runs for historical data intervals in scheduled DAGs. This may involve applying changes to the executor_config (e.g., updating Kubernetes worker resources or the Docker image). However, as described in [issue #26568](https://github.com/apache/airflow/issues/26568), the executor_config is saved in the database when the original DAG run is created. When a DAG run is cleared, this stored configuration is reused—so any updates to it are ignored.

A workaround is to delete the DAG run entirely. This forces the scheduler to re-parse the DAG definition and create a new DAG run with the updated executor_config.

**Reloading updated DAG structure**
If the DAG definition has changed (e.g., tasks have been added or removed), and you need to rerun historical DAG runs, there is a limitation in Airflow 3.0 when using versioned DAG bundles. In this case, clearing the DAG run won't reload the updated structure, since task instances are associated with the serialized DAG version that existed at the time of their creation.

If there are better ways to achieve these outcomes without deleting DAG runs, I’d appreciate any guidance on that.

### Related issues

[26568](https://github.com/apache/airflow/issues/26568)

### Are you willing to submit a PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start by tracing the existing DAG-run deletion entry points in the CLI and UI, then compare them with the Airflow 3.0 API behavior mentioned in the issue. Define how before and after logical-date bounds should work and verify that selected DAG runs are deleted without requiring additional scripting.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, data-engineering, frontend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.