apache / apache/datafusion-comet

Spark 4.2: native Iceberg REST catalog scan test fails under Comet

Open
#4,969 1 comment 0 reactions 1 assignee Claimed by @hsiang-c View on GitHub
area:scan bug priority:low spark 4.2
Dominant language
Scala
Stars
1.3k
Forks
375
Avg merge
2d 4h
Merged PRs (30d)
213

Description

The Comet test `CometIcebergNativeSuite`: "REST catalog with native Iceberg scan" is skipped on Spark 4.2 (`assume(!isSpark42Plus, ...)`).

No Iceberg spark-runtime is published for Spark 4.2, so the build reuses the 4.0 runtime; Spark 4.2 turned `connector.catalog.View` from an interface into a class, which affects the Iceberg integration (see the Iceberg availability probes already added for 4.2). This issue tracks re-enabling the native Iceberg REST catalog scan test on Spark 4.2.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.