Upgrade the supported Spark 3.5 version to 3.5.8
Open
enhancement
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### Description
The upgrade is blocked by a Delta-related issue: using the Gluten JAR built with Delta enabled triggers https://github.com/delta-io/delta/issues/4671.
This PR attempted the upgrade: https://github.com/apache/gluten/pull/12252. According to the Delta issue, the fix appears to be targeted for the unreleased Delta 3.3.3, so we may need to revisit this upgrade once that release is available.
Contributor guide
Research direction
Start by reviewing the attempted upgrade in apache/gluten#12252 and the linked Delta issue #4671. Revisit the change when Delta 3.3.3 is available, then verify that Gluten can support Spark 3.5.8 without the reported Delta-related failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala, spark
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100