Spark make_dt_interval/make_interval return NULL on overflow instead of erroring
Open
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
datafusion-spark's **make_dt_interval** / **make_interval** kernels use checked_add/checked_mul and append_null() on overflow.
Spark's **IntervalUtils.makeDayTimeInterval / makeInterval** use
Math.addExact/multiplyExact and throw SparkArithmeticException (INTERVAL_ARITHMETIC_OVERFLOW).
e.g. make_dt_interval(1000000000, 0, 0, 0) returns NULL here but errors in Spark.
Suggest erroring (or a configurable ANSI-style mode) to match Spark's documented behavior.
Contributor guide
Assessment
This issue has not been assessed yet.