apache / apache/gravitino

[Improvement] test more Iceberg versions in spark connector

Open
#3,778 2 comments 0 reactions 0 assignees View on GitHub
improvement
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 15h
Merged PRs (30d)
315

Description

### What would you like to be improved?

For now, Iceberg versions are compiled and test in 1.4.3, we'd better test 1.3, and 1.5 versions.

### How should we improve?

It's expensive to add another `-PicebergVersion` to run test, I suggest to test Iceberg 1.3 in Spark3.3, 1.4 in Spark3.4, 1.5 in spark3.5

Contributor guide

Open the contributing guide

Research direction

Start by locating the Spark connector's existing Iceberg-version test configuration and the -PicebergVersion path. Check how Spark 3.3, 3.4, and 3.5 profiles are defined, then verify that Iceberg 1.3, 1.4, and 1.5 are tested with the proposed Spark versions. Done means the version matrix runs without requiring a separate manual version argument.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spark
Domain
data-engineering, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.