Support unix_date and date_from_unix_date natively
- Dominant language
- Rust
- Stars
- 1.8k
- Forks
- 241
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 22
Description
**Is your feature request related to a problem? Please describe.**
Auron does not currently support native execution of Spark's `unix_date` and `date_from_unix_date` expressions. These expressions fall back to Spark evaluation.
**Describe the solution you'd like**
Add native support for both functions:
- `unix_date(date)` returns an integer day count relative to 1970-01-01.
- `date_from_unix_date(days)` converts that day count back to a date.
Preserve Spark's return types and null behavior. Add tests for zero, positive and negative day counts, integer boundaries, and round-trip conversions. Use column inputs in Spark regression tests and verify that both expressions execute natively.
**Describe alternatives you've considered**
Keep the existing Spark fallback.
**Additional context**
Both functions were introduced in Spark 3.1.0.
```sql
SELECT unix_date(DATE '1970-01-01'); -- 0
SELECT unix_date(DATE '1969-12-31'); -- -1
SELECT date_from_unix_date(0); -- 1970-01-01
SELECT date_from_unix_date(-1); -- 1969-12-31
```
Contributor guide
Research direction
Start by locating existing native date-expression implementations and the Spark regression tests for expression execution. Add coverage for zero, positive and negative day counts, integer boundaries, nulls, and round trips, then verify the column expressions execute natively with Spark-compatible return types.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, spark
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 72/100