Column statistics are not collected when the column name is escaped
- Dominant language
- Java
- Stars
- 9.2k
- Forks
- 3.5k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 132
Description
### Apache Iceberg version
1.7.1 (latest release)
### Query engine
None
### Please describe the bug 🐞
When an Iceberg has many columns, it calculates statistics only up to 100 columns (can be changed using config `write.metadata.metrics.max-inferred-column-defaults`).
When a column name is escaped, there is a bug that it tries to use the escaped column name when calculating the statistics while the metrics mode was saved using the original column name.
For example, I have a column `$event_time` which is escaped to `_x24event_time`.
When building the metrics mode, it uses the original name in [MetricsConfig.java:138](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/MetricsConfig.java#L138):

while when closing the parquet writer, it tries to get the metrics mode for `_x24event_time` in [MetricsUtil.java:118](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/MetricsUtil.java#L118):

when it doesn't find the metrics mode, it uses the default which is `none` so the stats are not collected.
### Willingness to contribute
- [ ] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [x] I cannot contribute a fix for this bug at this time
Contributor guide
Assessment
This issue has not been assessed yet.