apache / apache/arrow

[C++][Parquet] MinMax statistics for strings may be inaccurate after a merge

Open
#47,995 0 comments 0 reactions 0 assignees View on GitHub
Component: C++ Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

TypedStatistics::Megre function disregards column chunks when the minimum value is an empty string (represented as {null, 0}) due to the following code:
```c++
optional> CleanStatistic(
std::pair min_max, LogicalType::Type::type) {
if (min_max.first.ptr == nullptr || min_max.second.ptr == nullptr) {
return ::std::nullopt;
}
return min_max;
}
```
so both minimum and maximum values will be lost.

### Component(s)

C++

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.