airflow db clean fails with --dag-ids / --exclude-dag-ids
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 472
Description
### Apache Airflow version
3.2.2
### What happened and how to reproduce it?
When running `airflow db clean` with either `--dag-ids` or `--exclude-dag-ids`, the command fails with `RuntimeError: airflow db clean encountered errors on the following tables and did not clean them: ['asset_event', 'deadline', 'task_reschedule']`.
## Steps to reproduce
1. Run the cleanup command without DAG filtering:
```bash
airflow db clean \
--clean-before-timestamp "$(date -d '15 days ago' '+%Y-%m-%d')" \
--error-on-cleanup-failure \
--skip-archive \
--tables "_xcom_archive,asset_event,callback_request,celery_taskmeta,celery_tasksetmeta,import_error,job,session,sla_miss,task_reschedule,trigger,dag_run" \
-y
```
This completes successfully.
2. Run the same command with `--dag-ids`:
```bash
airflow db clean \
--clean-before-timestamp "$(date -d '15 days ago' '+%Y-%m-%d')" \
--dag-ids "your_dag_id" \
--error-on-cleanup-failure \
--skip-archive \
--tables "_xcom_archive,asset_event,callback_request,celery_taskmeta,celery_tasksetmeta,import_error,job,session,sla_miss,task_reschedule,trigger,dag_run" \
-y
```
3. The command fails while cleaning `asset_event`, `deadline`, and `task_reschedule`.
4. The same happens when using `--exclude-dag-ids`.
### What you think should happen instead?
`airflow db clean` should successfully respect `--dag-ids` / `--exclude-dag-ids`.
In particular, it should not fail on `asset_event`, `deadline`, or `task_reschedule` when the command is otherwise valid and the same run works without DAG filtering.
### Operating System
Debian GNU/Linux 12 (bookworm)
### Deployment
Official Apache Airflow Helm Chart
### Official Helm Chart version
1.21.0
### Anything else?
Notes:
- `--dag-ids` and `--exclude-dag-ids` are both documented by `airflow db clean --help`.
- The failure only occurs when those flags are used.
- The command works without DAG filtering.
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start at the `airflow db clean` CLI entry point and reproduce the failure with `--dag-ids` and `--exclude-dag-ids`, comparing both runs with the unfiltered command. Trace how those filters handle `asset_event`, `deadline`, and `task_reschedule`. Done means both filtered commands complete successfully and retain the intended DAG filtering behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100