Databricks: sensor for jobs
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Description
A sensor which allows waiting for completion of a Databricks job or task.
### Use case/motivation
We would like to make it easy to wait for a Databricks job, similar to how [ExternalTaskSensor](https://airflow.apache.org/docs/apache-airflow/2.0.2/_api/airflow/sensors/external_task/index.html) allows waiting for other Airflow tasks. Specifically, we’d like to support waiting for:
* A specific job run to complete
* A specific task of a job run to complete
The state of job runs can be queried via the [JobRunsGet](https://docs.databricks.com/dev-tools/api/latest/jobs.html#operation/JobsRunsGet.) API.
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Assessment
This issue has not been assessed yet.