Data: TestWriterMetrics.testMaxColumnsWithDefaultOverride does not verify bounds
- Dominant language
- Java
- Stars
- 9.2k
- Forks
- 3.5k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 129
Description
**Apache Iceberg version**
main @ 5f3d3c560
**Query engine**
None — engine-agnostic. Defined in `data/`, run by the Flink and Spark implementations of `TestWriterMetrics`.
**Please describe the bug**
`TestWriterMetrics.testMaxColumnsWithDefaultOverride()` (`data/src/test/java/org/apache/iceberg/io/TestWriterMetrics.java` line 344-352) does not verify what its name and comment describe: that setting `write.metadata.metrics.default` explicitly bypasses the max-inferred-column limit, so all 101 columns keep metrics.
- Line 346 reads lower bounds from `dataFile.upperBounds()`. `dataFile.lowerBounds()` is never called, so lower bounds go unchecked.
- The loop variable at line 347 is unused; both assertions hardcode `get(1)`. Only column 1 is checked, 101 times.
The test stays green even if the other 100 columns lose their metrics — the exact regression it exists to catch.
**Steps to reproduce**
Suppress the `configuredDefault != null` bypass in `MetricsConfig.from()` (`core/src/main/java/org/apache/iceberg/MetricsConfig.java` line 265) so `limitFieldIds` truncates to 100 columns.
Expected: the test fails. Actual: it passes on both ORC and Parquet.
**Additional context**
Sibling tests in the same file show the intended form: `testMaxColumnsBounded` (line 307) asserts the bounds map size, and `testMaxColumns` (line 241-269) uses its loop variable to check `lowerBounds()` and `upperBounds()` separately.
Introduced in #3959 (2022-02); the assertions have had this shape since.
Contributor guide
Research direction
Start in data/src/test/java/org/apache/iceberg/io/TestWriterMetrics.java at testMaxColumnsWithDefaultOverride, then compare testMaxColumns and testMaxColumnsBounded. Run the TestWriterMetrics tests for the Flink and Spark implementations; done means all 101 columns have both lower- and upper-bound metrics, and the test detects truncation when the MetricsConfig bypass is suppressed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 92/100