apache / apache/datafusion-comet
[Bug] Native make_interval overflows valid time components or loses seconds precision
- Dominant language
- Scala
- Stars
- 1.3k
- Forks
- 373
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 198
Description
## Describe the bug
Native `make_interval` delegates to DataFusion `SparkMakeInterval`. Its seconds argument is coerced from Spark `Decimal(18,6)` to `Float64`, and its time components are accumulated as i64 nanoseconds. This creates two Spark compatibility gaps:
- `Float64` coercion can lose microsecond precision.
- Nanosecond accumulation gives hours, minutes, and seconds a 1000x smaller range than Spark's microsecond accumulation, so valid Spark intervals can overflow natively.
## Steps to reproduce
```sql
SELECT make_interval(0, 0, 0, 0, 2562048);
SELECT make_interval(0, 0, 0, 0, 0, 0, 999999999.000001);
SELECT make_interval(1, 2, 3, 4, 0, 0, 123456789012.123456);
```
The hours query is valid in Spark but overflows the native nanosecond calculation. The second query loses one microsecond natively. The third returns NULL in non-ANSI mode or throws in ANSI mode natively, while Spark returns a valid interval.
The native overflow cutoffs include hours >= 2,562,048 and minutes >= 153,722,868; large seconds values have the same root cause.
## Expected behavior
Native `make_interval` should preserve `Decimal(18,6)` microsecond precision and support every hours, minutes, and seconds value Spark can represent.
## Additional context
Found while reviewing #5039. Until this is fixed, `CometMakeInterval` is marked Incompatible: the compatible default uses JVM codegen dispatch, and the native path requires explicit opt-in.
Review context: https://github.com/apache/datafusion-comet/pull/5039#discussion_r3659501474
Contributor guide
Assessment
This issue has not been assessed yet.