from_unixtime does not work for large values
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Describe the bug
It looks `from_unixtime` supports only -8334601211038 <= sec <= 8210266876799
### To Reproduce
```
$datafusion-cli
DataFusion CLI v48.0.0
> select from_unixtime(8210266876799, 'America/New_York');
+--------------------------------------------------------------+
| from_unixtime(Int64(8210266876799),Utf8("America/New_York")) |
+--------------------------------------------------------------+
| +262142-12-31T18:59:59-05:00 |
+--------------------------------------------------------------+
1 row(s) fetched.
Elapsed 0.039 seconds.
> select from_unixtime(8210266876799 + 1, 'America/New_York');
+----------------------------------------------------------------------------------------------------------------+
| from_unixtime(Int64(8210266876799) + Int64(1),Utf8("America/New_York")) |
+----------------------------------------------------------------------------------------------------------------+
| ERROR: Cast error: Failed to convert 8210266876800 to datetime for Timestamp(Second, Some("America/New_York")) |
+----------------------------------------------------------------------------------------------------------------+
1 row(s) fetched.
Elapsed 0.006 seconds.
> select from_unixtime(-8334601211038, 'America/New_York');
+---------------------------------------------------------------+
| from_unixtime(Int64(-8334601211038),Utf8("America/New_York")) |
+---------------------------------------------------------------+
| -262143-01-01T00:00:00-04:56 |
+---------------------------------------------------------------+
1 row(s) fetched.
Elapsed 0.001 seconds.
> select from_unixtime(-8334601211038 -1, 'America/New_York');
thread 'main' panicked at ...
Local time out of range for `NaiveDateTime`
```
### Expected behavior
We should be able to convert all `Int64` values
### Additional context
Spark can convert those large numbers
Contributor guide
Research direction
Start by reproducing the two boundary cases in datafusion-cli and trace the from_unixtime conversion for timezone-aware timestamps. Done means all Int64 values convert without a cast error or panic, including the values just beyond the reported limits.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100