apache / apache/airflow

Databricks: add SQL endpoint operators

Open
#21,377 12 comments 0 reactions 0 assignees View on GitHub
area:providers good first issue kind:feature provider:databricks
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Description

Operators to manipulate Databricks SQL endpoints.

### Use case/motivation

Support conveniently managing the lifecycle of Databricks SQL endpoints. This likely entails building one or more operators which talk to the [endpoints API](https://docs.databricks.com/sql/api/sql-endpoints.html) and support:

* Creating endpoints
* Deleting endpoints
* Starting endpoints
* Stopping endpoints

These operations map cleanly to individual API calls.

### Related issues

_No response_

### Are you willing to submit a PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Begin with the linked Databricks SQL endpoints API and map its lifecycle calls to Airflow's operator conventions. Identify the existing Databricks integration entry points and relevant operator tests before deciding whether one or several operators fit. Done means endpoint creation, deletion, starting, and stopping are supported and covered by tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.