apache / apache/datafusion-comet

Date-to-timestamp casts can overflow or panic for wide dates

Open
#5,456 0 comments 0 reactions 1 assignee Claimed by @sunchao View on GitHub
area:expressions bug correctness priority:critical
Dominant language
Scala
Stars
1.3k
Forks
373
Avg merge
2d 4h
Merged PRs (30d)
198

Description

### Describe the bug

After #5443 widened native `make_date` to Spark's Date32 range, composing it with a native timestamp cast exposes two unsafe consumers. This is independent of ANSI mode.

With Comet enabled, disable constant folding so these expressions execute rather than being evaluated by Spark during planning:

```sql
SET spark.sql.optimizer.excludedRules=org.apache.spark.sql.catalyst.optimizer.ConstantFolding;
SET spark.sql.ansi.enabled=false;
SET spark.sql.session.timeZone=UTC;

SELECT CAST(make_date(300000, 6, 15) AS TIMESTAMP_NTZ);
SELECT CAST(make_date(262143, 1, 1) AS TIMESTAMP);
```

For the first query, native `make_date` produces epoch day `108853388`. Multiplying by `86400000000` exceeds `i64`. Spark throws `ArithmeticException: long overflow`, but Comet's unchecked multiplication wraps to `-9041811350509551616` microseconds when overflow checks are disabled, as in release builds.

For the second query, Spark succeeds with `8210266876800000000` UTC microseconds. Comet panics at the chrono date addition with `NaiveDate + TimeDelta overflowed`, because chrono's positive year range ends at 262142.

The same paths are reachable with year/month/day columns from a table, without disabling constant folding. Both failures were reproduced in the native kernel and compared with Spark's date utilities. A fix needs checked arithmetic, a safe timezone conversion path, and non-folded composition tests.

### Affected version

Current main including #5443 (`e0ab0a6fe60c05bd679654f0201ebe88319cc3a7`).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.