Select version when re-running a dag/task
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Apache Airflow version
3.0.0
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
When testing new dags, especially long-running dags, it's really frustrating to have to re-run a long dag just to be able to re-run the last task (e.g. there was a bug in the last task). Dag versioning is great, but it undermines the concept of task atomicity because in order to re-run a single task, I potentially need to re-run an entire otherwise perfectly functioning dag.
### What you think should happen instead?
Users should be able to select a dag version when clearing a task if that task existed in previous versions.
### How to reproduce
Create a dag run with a bug in the last task. Fix the bug, then clear the last task instance.
### Operating System
n/a
### Versions of Apache Airflow Providers
_No response_
### Deployment
Astronomer
### Deployment details
_No response_
### Anything else?
This is arguably a feature request, but I'm considering it a bug since this functionality exists in Airflow 2.x
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start by tracing the task-clearing flow used when a task instance is cleared, then inspect how DAG versions are represented and selected during reruns. Done means a user can choose an earlier DAG version when clearing a task that existed in that version, without rerunning the entire DAG; add or update coverage for the described reproduction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100