apache / apache/iceberg

MR: IcebergSplit.getLocations discards cached block locations on repeated calls

Open Beginner friendly
#18,032 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
9.2k
Forks
3.5k
Avg merge
2d 16h
Merged PRs (30d)
129

Description

**Apache Iceberg version**
main @ 9d7b2c52f

**Query engine**
None — the bug is in `iceberg-mr` and is engine-agnostic.

**Please describe the bug**
`IcebergSplit.getLocations()` (`mr/src/main/java/org/apache/iceberg/mr/mapreduce/IcebergSplit.java` line 70-82) returns the computed block locations on the first call and `["*"]` on every call after that, even though nothing about the split changed.

The `transient locations` field and the `locations == null` check say the value is computed once and cached. The `else` branch defeats that: it runs whenever `locations` is already set and overwrites it with `ANYWHERE`. Hadoop's `InputSplit.getLocations()` contract expects repeated calls on an unchanged split to return the same value.

The `else` branch came from #1582, which kept a deserialized split on a worker node (where `conf` is null) from hitting an NPE. Before that, #1192 had `if (locations == null)` alone.

**Steps to reproduce**
Build splits with `iceberg.mr.locality` enabled (`InputFormatConfig.ConfigBuilder.preferLocality()`) and call `getLocations()` twice on the same split.

- Expected: both calls return the computed hosts, e.g. `["localhost"]`.
- Actual: the first returns `["localhost"]`, the second returns `["*"]`.

`TestIcebergInputFormats#testLocality` reproduces this without Docker.

**Additional context**
N/A

Contributor guide

Open the contributing guide

Research direction

Start with mr/src/main/java/org/apache/iceberg/mr/mapreduce/IcebergSplit.java, especially getLocations() around lines 70-82, and run TestIcebergInputFormats#testLocality. Reproduce the repeated-call behavior with locality enabled, then verify that an unchanged split returns the same computed hosts on both calls while deserialized splits without configuration remain supported.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
88/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.