skip_archive should actually skip archive in db clean command
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Body
Currently the delete process uses the archive table in the process of deletion. But we should refactor this so it doesn't need this and actually not create the archive table at all when user specifies this option.
The issue is when you are deleting a lot of rows, particularly if you have a low statement timeout, you may hit errors.
### Committer
- [X] I acknowledge that I am a maintainer/committer of the Apache Airflow project.
Contributor guide
Research direction
Start with the db clean command implementation and trace how the skip_archive option currently creates or uses the archive table during deletion. Done means that specifying skip_archive neither creates nor uses the archive table, while large deletions avoid the reported statement-timeout problem.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, database
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100