apache / apache/polaris

Create Spark 4 client

Open
#3,021 9 comments 2 reactions 1 assignee Claimed by @MonkeyCanCode View on GitHub
enhancement
Dominant language
Java
Stars
2.1k
Forks
522
Avg merge
2d 1h
Merged PRs (30d)
140

Description

### Is your feature request related to a problem? Please describe.

Currently only Spark 3.5 is supported for the Spark Client, we use the Spark client to write data in the Delta format, but we need to use Spark 4 in order to write queries like `SELECT * EXCEPT(...)` via the thrift server, as this is not being parsed correctly in earlier versions.

### Describe the solution you'd like

A version of the client jar built for Spark 4

### Describe alternatives you've considered

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.