"Queue up new tasks" equivalent in CLI / REST API
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 484
Description
### Description
Suggestion to add a new command in the CLI / REST API to reproduce the "Queue up new tasks" action that can be trigger within the web UI, with `start_date` and `end_date` parameters to allow running this action accross all dag runs lying a given period.
For the CLI, it could be `airflow dags queue_up_new_tasks` or `airflow dags reload` for instance.
### Use case/motivation
Right now, when we add new tasks to an existing dag containing existing dag runs, it is not straightforward to run these newly added tasks in the existing/previous dag runs.
Two workarounds :
- `airflow dags backfill` but not satisfying as it transforms "scheduled" existing dags run into "backfill" dag runs, which prevent us from interacting with them through the web UI (like clearing tasks, etc.)
- `airflow tasks render` followed by `airflow tasks clear` but it is not convenient as the `render` command is only working for one specific execution_date (or run_id) at a time.
Or maybe there are already solutions par that ?
### Related issues
https://github.com/apache/airflow/discussions/30435#discussioncomment-8856597
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start by comparing the web UI “Queue up new tasks” action with the existing `airflow dags backfill` and `airflow tasks render`/`clear` commands. Trace the CLI and REST API entry points involved, then clarify the date-range and existing-DAG-run behavior; done means the action is available through the requested interfaces without converting scheduled runs to backfills.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend-api-design, cli
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100