apache / apache/datafusion-comet

[Bug] Native make_interval overflows valid time components or loses seconds precision

Open
#5,131 0 comments 0 reactions 1 assignee Claimed by @peterxcli View on GitHub
area:expressions bug priority:medium
Dominant language
Scala
Stars
1.3k
Forks
373
Avg merge
2d 4h
Merged PRs (30d)
198

Description

## Describe the bug

Native `make_interval` delegates to DataFusion `SparkMakeInterval`. Its seconds argument is coerced from Spark `Decimal(18,6)` to `Float64`, and its time components are accumulated as i64 nanoseconds. This creates two Spark compatibility gaps:

- `Float64` coercion can lose microsecond precision.
- Nanosecond accumulation gives hours, minutes, and seconds a 1000x smaller range than Spark's microsecond accumulation, so valid Spark intervals can overflow natively.

## Steps to reproduce

```sql
SELECT make_interval(0, 0, 0, 0, 2562048);
SELECT make_interval(0, 0, 0, 0, 0, 0, 999999999.000001);
SELECT make_interval(1, 2, 3, 4, 0, 0, 123456789012.123456);
```

The hours query is valid in Spark but overflows the native nanosecond calculation. The second query loses one microsecond natively. The third returns NULL in non-ANSI mode or throws in ANSI mode natively, while Spark returns a valid interval.

The native overflow cutoffs include hours >= 2,562,048 and minutes >= 153,722,868; large seconds values have the same root cause.

## Expected behavior

Native `make_interval` should preserve `Decimal(18,6)` microsecond precision and support every hours, minutes, and seconds value Spark can represent.

## Additional context

Found while reviewing #5039. Until this is fixed, `CometMakeInterval` is marked Incompatible: the compatible default uses JVM codegen dispatch, and the native path requires explicit opt-in.

Review context: https://github.com/apache/datafusion-comet/pull/5039#discussion_r3659501474

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.