Add non-UTC timestamp coverage for native datediff
- Dominant language
- Rust
- Stars
- 1.8k
- Forks
- 241
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 21
Description
**Is your feature request related to a problem? Please describe.**
PR #2502 added native support for `datediff(endDate, startDate)`, but the current regression coverage only uses `DATE` inputs.
Spark also accepts `TIMESTAMP` inputs and converts them to dates using the session time zone. The native path should cover cases where the local date differs from the UTC date.
**Describe the solution you'd like**
Add a regression case to `AuronFunctionSuite` that:
- uses `TIMESTAMP` columns;
- sets `spark.sql.session.timeZone` to a non-UTC zone such as `America/Los_Angeles`;
- uses values around a local/UTC date boundary;
- verifies that the result matches Spark and that the native operator is used.
For example:
```sql
CREATE TABLE t1(end_ts TIMESTAMP, start_ts TIMESTAMP) USING parquet;
INSERT INTO t1 VALUES
(TIMESTAMP '1970-01-01 16:30:00', TIMESTAMP '1970-01-01 00:30:00');
SELECT datediff(end_ts, start_ts) FROM t1;
```
With America/Los_Angeles as the session time zone, the expected result is 0.
**Describe alternatives you've considered**
Keep the existing DATE-only coverage. The implementation remains covered for its core calculation, but the timestamp-to-date conversion path is not protected against regressions.
**Additional context**
This is a follow-up to #2501 and #2502.
The issue only tracks regression coverage and does not assume that the current implementation has a correctness problem.
Contributor guide
Research direction
Start with the existing datediff coverage in AuronFunctionSuite and inspect how it verifies native operator use. Add a TIMESTAMP case using a non-UTC spark.sql.session.timeZone and values around a date boundary, then run the suite to confirm the native result matches Spark and returns 0 for the example.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, spark, sql
- Domain
- data-engineering, testing
- Issue type
- Feature
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 82/100