Add a new ExternalAPITaskSensor to monitor external DAGs via Airflow REST API
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Description
I'm proposing to add a new ExternalAPITaskSensor to the Apache Airflow project. This sensor leverages the Airflow REST API to monitor the status of an external DAG or task. It can be used to check the completion of another DAG or task before proceeding with the execution of the current task, providing better orchestration across DAGs.
### Use case/motivation
The new sensor provides the ability to monitor the status of external DAGs using the Airflow REST API, which is particularly useful in a distributed environment where multiple Airflow instances are running.
### Related issues
_No response_
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start by reviewing existing Airflow sensors and the Airflow REST API endpoints for external DAG and task status. Define how the new sensor should identify and monitor a DAG or task across Airflow instances, and consider the test coverage needed to show that it waits for completion before allowing the current task to proceed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend-api-design, data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100