add option to set DAG level Dataset outlets
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 484
Description
### Description
add a new argument `outlets` to the DAG constructor with the same effect as the BaseOperator's outlets argument when the full dag run is successfull
### Use case/motivation
use case:
I'm currently maintaining several DAGs with "Dataset Scheduling", where each DAG handles a specific dataset and declares that it is ready for downstream DAGs by an EmptyOperator with an `outlets` value
this means I need to fan-in all the leafs of each DAG:

suggestion:
add an `outlets` argument to the `DAG` constructor to mark the Dataset(s) as ready when the dag run is successful
### Related issues
_No response_
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start at the DAG constructor and compare the requested behavior with the existing BaseOperator outlets argument and Dataset Scheduling flow. Trace how a successful full DAG run marks outlets ready, then verify the behavior with the relevant existing scheduling tests; done means DAG-level outlets are emitted only after successful runs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100