apache / apache/datafusion

Spark make_dt_interval/make_interval return NULL on overflow instead of erroring

Open
#22,401 3 comments 0 reactions 1 assignee Claimed by @kumarUjjawal View on GitHub
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

datafusion-spark's **make_dt_interval** / **make_interval** kernels use checked_add/checked_mul and append_null() on overflow.

Spark's **IntervalUtils.makeDayTimeInterval / makeInterval** use
Math.addExact/multiplyExact and throw SparkArithmeticException (INTERVAL_ARITHMETIC_OVERFLOW).
e.g. make_dt_interval(1000000000, 0, 0, 0) returns NULL here but errors in Spark.

Suggest erroring (or a configurable ANSI-style mode) to match Spark's documented behavior.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.