apache / apache/datafusion-comet
Spark 4.2: native Iceberg REST catalog scan test fails under Comet
Open
area:scan
bug
priority:low
spark 4.2
- Dominant language
- Scala
- Stars
- 1.3k
- Forks
- 375
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 213
Description
The Comet test `CometIcebergNativeSuite`: "REST catalog with native Iceberg scan" is skipped on Spark 4.2 (`assume(!isSpark42Plus, ...)`).
No Iceberg spark-runtime is published for Spark 4.2, so the build reuses the 4.0 runtime; Spark 4.2 turned `connector.catalog.View` from an interface into a class, which affects the Iceberg integration (see the Iceberg availability probes already added for 4.2). This issue tracks re-enabling the native Iceberg REST catalog scan test on Spark 4.2.
Contributor guide
Assessment
This issue has not been assessed yet.