[R] Microsecond is not sufficient unit for POSIXct
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 91
Description
I believe the `POSIXct` type or R currently corresponds to the Arrow `timestamp[us, tz=UTC]` type.
```r
lubridate::as_datetime(0) |> arrow::infer_type()
#> Timestamp
#> timestamp[us, tz=UTC]
```
```r
lubridate::as_datetime("1970-01-01 00:00:00.0000001") |>
arrow::arrow_table(x = _)
#> Table
#> 1 rows x 1 columns
#> $x
```
```r
df_a <- lubridate::as_datetime("1970-01-01 00:00:00.0000001") |>
arrow::arrow_table(x = _) |>
as.data.frame()
df_b <- lubridate::as_datetime("1970-01-01 00:00:00.0000001") |>
tibble::tibble(x = _)
waldo::compare(df_a, df_b)
#> `old$x`: "1970-01-01"
#> `new$x`: "1970-01-01 00:00:00"
```
However, as shown below, POSIXct may hold data finer than a microsecond.
```r
lubridate::as_datetime(0.000000001) |> as.numeric()
#> [1] 1e-09
lubridate::as_datetime("1970-01-01 00:00:00.0000001") |> as.numeric()
#> [1] 1.192093e-07
```
I don't know why it is currently set in microseconds, but is there any reason not to set it in nanoseconds?
**Reporter**: [SHIMA Tatsuya](https://issues.apache.org/jira/browse/ARROW-17424) / @eitsupi
#### Related issues:
- [[R] Convert a column as a sub-second timestamp from CSV file with the `T` col type option](https://github.com/apache/arrow/issues/31061) (relates to)
- [[R] `lubridate::as_datetime()` in dplyr query should be able to handle time in sub seconds](https://github.com/apache/arrow/issues/32694) (is related to)
**Note**: *This issue was originally created as [ARROW-17424](https://issues.apache.org/jira/browse/ARROW-17424). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*
Contributor guide
Research direction
Start with the lubridate::as_datetime(), arrow::infer_type(), and arrow_table() examples in the issue, then trace how R POSIXct values are converted to Arrow timestamps. Compare values with sub-microsecond precision and confirm that the chosen timestamp unit preserves them without changing ordinary datetime behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100