apache / apache/airflow

airflow db clean fails with --dag-ids / --exclude-dag-ids

Open
#69,030 2 comments 2 reactions 0 assignees View on GitHub
area:CLI area:MetaDB kind:bug needs-triage
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Apache Airflow version

3.2.2

### What happened and how to reproduce it?

When running `airflow db clean` with either `--dag-ids` or `--exclude-dag-ids`, the command fails with `RuntimeError: airflow db clean encountered errors on the following tables and did not clean them: ['asset_event', 'deadline', 'task_reschedule']`.

## Steps to reproduce

1. Run the cleanup command without DAG filtering:

```bash
airflow db clean \
--clean-before-timestamp "$(date -d '15 days ago' '+%Y-%m-%d')" \
--error-on-cleanup-failure \
--skip-archive \
--tables "_xcom_archive,asset_event,callback_request,celery_taskmeta,celery_tasksetmeta,import_error,job,session,sla_miss,task_reschedule,trigger,dag_run" \
-y
```

This completes successfully.

2. Run the same command with `--dag-ids`:

```bash
airflow db clean \
--clean-before-timestamp "$(date -d '15 days ago' '+%Y-%m-%d')" \
--dag-ids "your_dag_id" \
--error-on-cleanup-failure \
--skip-archive \
--tables "_xcom_archive,asset_event,callback_request,celery_taskmeta,celery_tasksetmeta,import_error,job,session,sla_miss,task_reschedule,trigger,dag_run" \
-y
```

3. The command fails while cleaning `asset_event`, `deadline`, and `task_reschedule`.

4. The same happens when using `--exclude-dag-ids`.

### What you think should happen instead?

`airflow db clean` should successfully respect `--dag-ids` / `--exclude-dag-ids`.

In particular, it should not fail on `asset_event`, `deadline`, or `task_reschedule` when the command is otherwise valid and the same run works without DAG filtering.

### Operating System

Debian GNU/Linux 12 (bookworm)

### Deployment

Official Apache Airflow Helm Chart

### Official Helm Chart version

1.21.0

### Anything else?

Notes:
- `--dag-ids` and `--exclude-dag-ids` are both documented by `airflow db clean --help`.
- The failure only occurs when those flags are used.
- The command works without DAG filtering.

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start at the `airflow db clean` CLI entry point and reproduce the failure with `--dag-ids` and `--exclude-dag-ids`, comparing both runs with the unfiltered command. Trace how those filters handle `asset_event`, `deadline`, and `task_reschedule`. Done means both filtered commands complete successfully and retain the intended DAG filtering behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.