apache / apache/kyuubi

[Umbrella] Support Apache Arrow Flight SQL

Open
#3,356 0 comments 0 reactions 0 assignees View on GitHub
kind:umbrella priority:major
Dominant language
Scala
Stars
2.4k
Forks
1k
PR merge metrics
No merged PRs in 30d

Description

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

### Search before asking

- [X] I have searched in the [issues](https://github.com/apache/incubator-kyuubi/issues) and found no similar issues.

### Describe the proposal

Apache Arrow Flight SQL is a future-proof tech in the SQL system area, we can expect huge performance improvement in data transmission compared to JDBC/ODBC(based on Hive Thrift protocol) and a fast-growing ecosystem.

Flight SQL means database servers can implement a standard interface that is designed around Apache Arrow and columnar data from the start. Just like how Arrow provides a standard in-memory format, Flight SQL saves developers from having to design and implement an entirely new wire protocol.

[1] https://arrow.apache.org/blog/2022/02/16/introducing-arrow-flight-sql/
[2] https://www.dremio.com/subsurface/arrow-flight-and-flight-sql-accelerating-data-movement/

### Task list

- #3364

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start by reviewing task #3364 and the Apache Arrow Flight SQL references linked in this issue. The proposal is an umbrella for supporting a new SQL interface; completion would require the scope to be broken into concrete implementation tasks and Flight SQL support to be established in Kyuubi.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala, sql
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.