Add GCS, Azure Support for AnalyticsOperator
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 472
Description
As part of [#62232](https://github.com/apache/airflow/pull/62232), support was added for **AWS (S3)** and **LocalFileSystem** object stores when connecting from DataFusion.
However, DataFusion supports several additional object store backends that are not yet covered. According to the [DataFusion Object Store documentation](https://datafusion.apache.org/python/autoapi/datafusion/object_store/index.html), the following are also available:
- **Google Cloud Storage (GCS)**
- **Azure Blob Storage / Azure Data Lake Storage (ADLS)**
### Motivation
Extending object store support will allow users to seamlessly query data residing in a wider range of storage backends directly through the Airflow AnalyticsOperator.
### Requested Changes
- Add support for **Google Cloud Storage (GCS)** object store integration
- Add support for **Azure Blob Storage** object store integration
### Related
- PR: https://github.com/apache/airflow/pull/62232
- DataFusion Object Store API Reference: https://datafusion.apache.org/python/autoapi/datafusion/object_store/index.html
Contributor guide
Assessment
This issue has not been assessed yet.