apache / apache/airflow

skip_archive should actually skip archive in db clean command

Open
#42,003 9 comments 1 reaction 0 assignees View on GitHub
area:db-migrations good first issue kind:feature kind:meta
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Body

Currently the delete process uses the archive table in the process of deletion. But we should refactor this so it doesn't need this and actually not create the archive table at all when user specifies this option.

The issue is when you are deleting a lot of rows, particularly if you have a low statement timeout, you may hit errors.

### Committer

- [X] I acknowledge that I am a maintainer/committer of the Apache Airflow project.

Contributor guide

Open the contributing guide

Research direction

Start with the db clean command implementation and trace how the skip_archive option currently creates or uses the archive table during deletion. Done means that specifying skip_archive neither creates nor uses the archive table, while large deletions avoid the reported statement-timeout problem.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.