apache / apache/arrow

[R] Empty character attributes not stored

Open
#29,233 7 comments 0 reactions 0 assignees View on GitHub
Component: R Priority: Critical Status: needs champion Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

Date-times in the POSIXct format have a 'tzone' attribute that by default is set to "", an empty character vector (not NULL) when created.

This however is not stored in the Arrow feather file. When the file is read back, the original and restored dataframes are not identical as per the below reprex.

I am thinking that this should not be the intention? My workaround at the moment is making a check when reading back to write the empty string if the tzone attribute does not exist.

Just to confirm, the attribute is stored correctly when it is not empty.

Thanks.
```java

``` r
dates <- as.POSIXct(c("2020-01-01", "2020-01-02", "2020-01-02"))
attributes(dates)
#> $class
#> [1] "POSIXct" "POSIXt"
#>
#> $tzone
#> [1] ""

values <- c(1:3)
original <- data.frame(dates, values)
original
#> dates values
#> 1 2020-01-01 1
#> 2 2020-01-02 2
#> 3 2020-01-02 3

tempfile <- tempfile()
arrow::write_feather(original, tempfile)

restored <- arrow::read_feather(tempfile)

identical(original, restored)
#> [1] FALSE

waldo::compare(original, restored)
#> `attr(old$dates, 'tzone')` is a character vector ('')
#> `attr(new$dates, 'tzone')` is absent

unlink(tempfile)
```
```
 

**Environment**: Ubuntu 20.04 R 4.1 release
**Reporter**: [Charlie Gao](https://issues.apache.org/jira/browse/ARROW-13588)

**Note**: *This issue was originally created as [ARROW-13588](https://issues.apache.org/jira/browse/ARROW-13588). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Research direction

Start by running the R reprex in the issue to reproduce the Feather round-trip mismatch. Then trace how R attributes are written and restored during Feather I/O. Done means an empty POSIXct tzone attribute survives read-back and identical(original, restored) returns TRUE.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
data-engineering
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.