CAST statistics can produce incorrect MIN/MAX results
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Describe the bug
Casting string min/max statistics to integers does not preserve their ordering. Aggregate optimization can therefore return incorrect results.
### To Reproduce
```sql
SET datafusion.execution.target_partitions = 1;
SET datafusion.execution.collect_statistics = true;
COPY (SELECT * FROM (VALUES ('1'), ('100'), ('2')) AS t(a))
TO '/tmp/cast_statistics.parquet' STORED AS PARQUET;
CREATE EXTERNAL TABLE t STORED AS PARQUET
LOCATION '/tmp/cast_statistics.parquet';
SELECT MIN(CAST(a AS INT)), MAX(CAST(a AS INT)) FROM t;
```
Actual: `1, 2`. Expected: `1, 100`. The same issue occurs with BIGINT.
### Expected behavior
Only propagate CAST statistics when the conversion is proven safe for the supplied bounds.
### Additional context
found this when work on https://github.com/apache/datafusion/issues/25220
Contributor guide
Assessment
This issue has not been assessed yet.