apache / apache/datafusion-comet
Date-to-timestamp casts can overflow or panic for wide dates
- Dominant language
- Scala
- Stars
- 1.3k
- Forks
- 373
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 198
Description
### Describe the bug
After #5443 widened native `make_date` to Spark's Date32 range, composing it with a native timestamp cast exposes two unsafe consumers. This is independent of ANSI mode.
With Comet enabled, disable constant folding so these expressions execute rather than being evaluated by Spark during planning:
```sql
SET spark.sql.optimizer.excludedRules=org.apache.spark.sql.catalyst.optimizer.ConstantFolding;
SET spark.sql.ansi.enabled=false;
SET spark.sql.session.timeZone=UTC;
SELECT CAST(make_date(300000, 6, 15) AS TIMESTAMP_NTZ);
SELECT CAST(make_date(262143, 1, 1) AS TIMESTAMP);
```
For the first query, native `make_date` produces epoch day `108853388`. Multiplying by `86400000000` exceeds `i64`. Spark throws `ArithmeticException: long overflow`, but Comet's unchecked multiplication wraps to `-9041811350509551616` microseconds when overflow checks are disabled, as in release builds.
For the second query, Spark succeeds with `8210266876800000000` UTC microseconds. Comet panics at the chrono date addition with `NaiveDate + TimeDelta overflowed`, because chrono's positive year range ends at 262142.
The same paths are reachable with year/month/day columns from a table, without disabling constant folding. Both failures were reproduced in the native kernel and compared with Spark's date utilities. A fix needs checked arithmetic, a safe timezone conversion path, and non-folded composition tests.
### Affected version
Current main including #5443 (`e0ab0a6fe60c05bd679654f0201ebe88319cc3a7`).
Contributor guide
Assessment
This issue has not been assessed yet.