[Improvement] IcebergRewriteDataFilesJob Array index out of bounds error
- Dominant language
- Java
- Stars
- 3.2k
- Forks
- 935
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 339
Description
### What would you like to be improved?
iceberg version:1.4.3
gravitino version:1.3.0
I created a template task for merging small files using iceberg. The small files were successfully merged, but when gravitino attempted to obtain the results, it reported an error. Eventually, the task on the web interface also showed as "failed".
The iceberg Low version is recommended for adaptation.
### How should we improve?
example:
# IcebergRewriteDataFilesJob.java L176-201
if (results.length > 0) {
Row result = results[0];
int size = result.size();
// Iceberg rewrite_data_files return schema varies by version:
// 1.4.x -> 3 columns (rewritten, added, rewritten_bytes)
// 1.5+/1.6+ -> 4 columns (adds failed_data_files)
// newer -> 5 columns (adds removed_delete_files)
// Print only the columns actually present to stay compatible with all versions.
StringBuilder sb = new StringBuilder("Rewrite Data Files Results:\n");
if (size > 0) {
sb.append(" Rewritten data files: ").append(result.getInt(0)).append("\n");
}
if (size > 1) {
sb.append(" Added data files: ").append(result.getInt(1)).append("\n");
}
if (size > 2) {
sb.append(" Rewritten bytes: ").append(result.getLong(2)).append("\n");
}
if (size > 3) {
sb.append(" Failed data files: ").append(result.getInt(3)).append("\n");
}
if (size > 4) {
sb.append(" Removed delete files: ").append(result.getInt(4)).append("\n");
}
System.out.print(sb);
}
Contributor guide
Research direction
Start with IcebergRewriteDataFilesJob.java around lines 176-201 and inspect how the rewrite_data_files result schema is read for Iceberg 1.4.3. Verify the job handles the version's returned columns without an array index error, and confirm the task completes successfully across the supported result schemas.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 82/100