[R] Error when saving attached data.frame containing nested data.frame (character versus all NA logical column)
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
### Describe the bug, including details regarding any error messages, version, and platform.
When saving the attached rds file to a parquet file, I get the error as in the repress `Error: Invalid: Problem with column 3 (au_orcid): Invalid: Expecting a character vector`.
The problem seems to be that one column only contains NA, and these are interpreted as logical by R, while arrow expects a character vector.
```
arrow 15.0.1 2024-03-19 [1] https://apache.r-universe.dev (R 4.3.3)
setting value
version R version 4.3.3 (2024-02-29)
os macOS Sonoma 14.4
system aarch64, darwin20
ui X11
language (EN)
collate en_US.UTF-8
ctype en_US.UTF-8
tz Europe/Zurich
date 2024-03-21
pandoc 3.1.12.3 @ /opt/homebrew/bin/pandoc
```
``` r
library(arrow)
#>
#> Attaching package: ‘arrow’
#> The following object is masked from ‘package:utils’:
#>
#> timestamp
data <- readRDS(“problem.rds”)
arrow::write_dataset(
data,
path = “~/problem”,
partitioning = c(“publication_year”, “page”),
format = “parquet”
)
#> Error: Invalid: Problem with column 3 (au_orcid): Invalid: Expecting a character vector
```
Created on 2024-03-21 with [reprex v2.1.0](https://reprex.tidyverse.org/)
[problem.rds.zip](https://github.com/apache/arrow/files/14696782/problem.rds.zip)
### Component(s)
R
Contributor guide
Research direction
Start by loading the attached problem.rds in R and reproducing the failure with arrow::write_dataset(), using the issue's partitioning and parquet settings. Trace how the nested data.frame and the all-NA au_orcid column are converted, then verify that the same example saves successfully without the character-vector error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100