apache / apache/datafusion-comet

[Feature] Support Spark expression: date_add_interval

Open
#3,086 0 comments 0 reactions 0 assignees View on GitHub
enhancement temporal expressions
Dominant language
Scala
Stars
1.3k
Forks
373
Avg merge
2d 6h
Merged PRs (30d)
190

Description

## What is the problem the feature request solves?

> **Note:** This issue was generated with AI assistance. The specification details have been extracted from Spark documentation and may need verification.

Comet does not currently support the Spark `date_add_interval` function, causing queries using this function to fall back to Spark's JVM execution instead of running natively on DataFusion.

The `DateAddInterval` expression adds a calendar interval to a date value and returns the resulting date. It supports both ANSI-compliant mode for strict error handling and optimized evaluation paths based on the interval's microsecond component.

Supporting this expression would allow more Spark workloads to benefit from Comet's native acceleration.

## Describe the potential solution

### Spark Specification

**Syntax:**
```sql
date_column + interval_expression
```

**Arguments:**
| Argument | Type | Description |
|----------|------|-------------|
| start | DateType | The starting date to which the interval will be added |
| interval | CalendarIntervalType | The calendar interval containing months, days, and microseconds |
| timeZoneId | Option[String] | Optional timezone identifier for timestamp conversions (defaults to None) |
| ansiEnabled | Boolean | Flag indicating whether ANSI mode is enabled (defaults to SQLConf setting) |

**Return Type:** `DateType` - Returns a date value representing the sum of the input date and interval.

**Supported Data Types:**
- **Input**: `DateType` for the start date, `CalendarIntervalType` for the interval
- **Output**: `DateType`

**Edge Cases:**
- **Null handling**: Null-intolerant - returns null if either input is null
- **ANSI mode**: Throws `IllegalArgumentException` for invalid date arithmetic operations
- **Microsecond precision**: Automatically handles conversion between date and timestamp representations based on interval precision
- **Timezone sensitivity**: Uses specified timezone for intermediate timestamp calculations when microseconds are present
- **Overflow**: Delegates overflow handling to underlying `DateTimeUtils` methods

**Examples:**
```sql
-- Add 1 month to a date
SELECT DATE '2023-01-15' + INTERVAL '1' MONTH;

-- Add complex interval to date
SELECT DATE '2023-01-15' + INTERVAL '2 months 10 days';
```

```scala
// DataFrame API usage
import org.apache.spark.sql.functions._

df.select(col("date_column") + expr("INTERVAL '1' MONTH"))

// Using interval literal
df.select(col("date_column") + lit(CalendarInterval.fromString("1 month 5 days")))
```

### Implementation Approach

See the [Comet guide on adding new expressions](https://datafusion.apache.org/comet/contributor-guide/adding_a_new_expression.html) for detailed instructions.

1. **Scala Serde**: Add expression handler in `spark/src/main/scala/org/apache/comet/serde/`
2. **Register**: Add to appropriate map in `QueryPlanSerde.scala`
3. **Protobuf**: Add message type in `native/proto/src/proto/expr.proto` if needed
4. **Rust**: Implement in `native/spark-expr/src/` (check if DataFusion has built-in support first)

## Additional context

**Difficulty:** Medium
**Spark Expression Class:** `org.apache.spark.sql.catalyst.expressions.DateAddInterval`

**Related:**
- `DatetimeSub` - Subtracts intervals from dates
- `TimestampAddInterval` - Adds intervals to timestamps
- `CalendarInterval` - Represents calendar intervals with months, days, and microseconds

---
*This issue was auto-generated from Spark reference documentation.*

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.