apache / apache/gravitino

[Improvement] IcebergRewriteDataFilesJob Array index out of bounds error

Open Beginner friendly
#12,479 0 comments 0 reactions 0 assignees View on GitHub
improvement
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 17h
Merged PRs (30d)
339

Description

### What would you like to be improved?

iceberg version:1.4.3
gravitino version:1.3.0

I created a template task for merging small files using iceberg. The small files were successfully merged, but when gravitino attempted to obtain the results, it reported an error. Eventually, the task on the web interface also showed as "failed".

Image

Image

Image

The iceberg Low version is recommended for adaptation.

### How should we improve?

example:
# IcebergRewriteDataFilesJob.java L176-201

if (results.length > 0) {
Row result = results[0];
int size = result.size();
// Iceberg rewrite_data_files return schema varies by version:
// 1.4.x -> 3 columns (rewritten, added, rewritten_bytes)
// 1.5+/1.6+ -> 4 columns (adds failed_data_files)
// newer -> 5 columns (adds removed_delete_files)
// Print only the columns actually present to stay compatible with all versions.
StringBuilder sb = new StringBuilder("Rewrite Data Files Results:\n");
if (size > 0) {
sb.append(" Rewritten data files: ").append(result.getInt(0)).append("\n");
}
if (size > 1) {
sb.append(" Added data files: ").append(result.getInt(1)).append("\n");
}
if (size > 2) {
sb.append(" Rewritten bytes: ").append(result.getLong(2)).append("\n");
}
if (size > 3) {
sb.append(" Failed data files: ").append(result.getInt(3)).append("\n");
}
if (size > 4) {
sb.append(" Removed delete files: ").append(result.getInt(4)).append("\n");
}
System.out.print(sb);
}

Contributor guide

Open the contributing guide

Research direction

Start with IcebergRewriteDataFilesJob.java around lines 176-201 and inspect how the rewrite_data_files result schema is read for Iceberg 1.4.3. Verify the job handles the version's returned columns without an array index error, and confirm the task completes successfully across the supported result schemas.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
82/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.