[Feature] Support Spark 4.1 with Paimon
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Motivation
Spark 4.1 is out and is [available](https://spark.apache.org/downloads.html) that contains some cool features like New Arrow-native UDF and VARIANT types.
### Solution
Similar to other Spark version, we can add support for 4.1 as well.
### Anything else?
_No response_
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue names no files or tests. Compare the repository's existing Spark-version support to identify the relevant build and compatibility entry points, then verify that Spark 4.1 is supported alongside the other Spark versions and that the existing integration checks pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100