MR: IcebergSplit.getLocations discards cached block locations on repeated calls
- Dominant language
- Java
- Stars
- 9.2k
- Forks
- 3.5k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 129
Description
**Apache Iceberg version**
main @ 9d7b2c52f
**Query engine**
None — the bug is in `iceberg-mr` and is engine-agnostic.
**Please describe the bug**
`IcebergSplit.getLocations()` (`mr/src/main/java/org/apache/iceberg/mr/mapreduce/IcebergSplit.java` line 70-82) returns the computed block locations on the first call and `["*"]` on every call after that, even though nothing about the split changed.
The `transient locations` field and the `locations == null` check say the value is computed once and cached. The `else` branch defeats that: it runs whenever `locations` is already set and overwrites it with `ANYWHERE`. Hadoop's `InputSplit.getLocations()` contract expects repeated calls on an unchanged split to return the same value.
The `else` branch came from #1582, which kept a deserialized split on a worker node (where `conf` is null) from hitting an NPE. Before that, #1192 had `if (locations == null)` alone.
**Steps to reproduce**
Build splits with `iceberg.mr.locality` enabled (`InputFormatConfig.ConfigBuilder.preferLocality()`) and call `getLocations()` twice on the same split.
- Expected: both calls return the computed hosts, e.g. `["localhost"]`.
- Actual: the first returns `["localhost"]`, the second returns `["*"]`.
`TestIcebergInputFormats#testLocality` reproduces this without Docker.
**Additional context**
N/A
Contributor guide
Research direction
Start with mr/src/main/java/org/apache/iceberg/mr/mapreduce/IcebergSplit.java, especially getLocations() around lines 70-82, and run TestIcebergInputFormats#testLocality. Reproduce the repeated-call behavior with locality enabled, then verify that an unchanged split returns the same computed hosts on both calls while deserialized splits without configuration remain supported.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 88/100