`UnwrapCastInComparison` produces incorrect results
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Describe the bug
I found that `UnwrapCastInComparison` always assumes the cast operation can succeed, but when it cannot, it results in incorrect optimization results.
### To Reproduce
Run query in CLI (compiled from the latest main: https://github.com/apache/datafusion/commit/f77579108d1dc0285636fbfb24507d2bfca66446)
```sh
DataFusion CLI v44.0.0
> with t as (select 1000000 as a) select try_cast(a as smallint) > 1 from t;
+----------------+
| t.a > Int64(1) |
+----------------+
| true |
+----------------+
1 row(s) fetched.
Elapsed 0.008 seconds.
> with t as (select 1000000 as a) select cast(a as smallint) > 1 from t;
+----------------+
| t.a > Int64(1) |
+----------------+
| true |
+----------------+
1 row(s) fetched.
Elapsed 0.007 seconds.
```
### Expected behavior
When optimizations are disabled, the above queries will produce different results, which are correct.
```sh
> set datafusion.optimizer.max_passes=0;
0 row(s) fetched.
Elapsed 0.003 seconds.
> with t as (select 1000000 as a) select try_cast(a as smallint) > 1 from t;
+----------------+
| t.a > Int64(1) |
+----------------+
| NULL |
+----------------+
1 row(s) fetched.
Elapsed 0.006 seconds.
> with t as (select 1000000 as a) select cast(a as smallint) > 1 from t;
Arrow error: Cast error: Can't cast value 1000000 to type Int16
```
### Additional context
I don't think this is a very urgent bug because both Spark and DuckDB have similar issues.
Contributor guide
Assessment
This issue has not been assessed yet.