apache / apache/airflow

Add db cleanup function for easier debugging and testing of dags

Open
#63,107 2 comments 0 reactions 0 assignees View on GitHub
area:MetaDB kind:feature needs-triage
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Description

I think that a function that automatically truncates the `Airflow`'s can be beneficial for us to help us combine and manage the state of airflow with [`dag.test` and debugging dags](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/debug.html).

### Use case/motivation

I want to have a function that I can call using my python code (mostly in my tests) to truncate the whole airflow db locally, it would only work in unit tests mode and on sqlite only (to avoid deleting production db).

### Related issues

_No response_

### Are you willing to submit a PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start by reading the dag.test and debugging DAG documentation linked in the issue, then identify the existing Airflow test-database setup and SQLite-only safeguards. Done means a callable function can safely truncate the local Airflow database in unit-test mode and support repeatable DAG tests and debugging without affecting production databases.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, sqlite
Domain
data-engineering, databases, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.