apache / apache/datafusion-comet
change default Maven profile to Spark 4.0
Open
area:ci
priority:low
- Dominant language
- Scala
- Stars
- 1.3k
- Forks
- 373
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 198
Description
#4140 changed the default profile to Spark 4.1, but I think that was premature considering we don't have Iceberg support on 4.1 yet. Iceberg 1.11 only released last week with Spark 4.1 support, and #4381 tracks Comet supporting Iceberg 1.11.
In the meantime, I manually set my environment to 4.0 locally for development, and would prefer others do the same to hopefully avoid breaking Iceberg compatibility accidentally.
Contributor guide
Assessment
This issue has not been assessed yet.