apache / apache/arrow

[R] Error when saving attached data.frame containing nested data.frame (character versus all NA logical column)

Open
#40,711 1 comment 0 reactions 0 assignees View on GitHub
Component: R Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

When saving the attached rds file to a parquet file, I get the error as in the repress `Error: Invalid: Problem with column 3 (au_orcid): Invalid: Expecting a character vector`.

The problem seems to be that one column only contains NA, and these are interpreted as logical by R, while arrow expects a character vector.

```
arrow 15.0.1 2024-03-19 [1] https://apache.r-universe.dev (R 4.3.3)

setting value
version R version 4.3.3 (2024-02-29)
os macOS Sonoma 14.4
system aarch64, darwin20
ui X11
language (EN)
collate en_US.UTF-8
ctype en_US.UTF-8
tz Europe/Zurich
date 2024-03-21
pandoc 3.1.12.3 @ /opt/homebrew/bin/pandoc
```

``` r
library(arrow)
#>
#> Attaching package: ‘arrow’
#> The following object is masked from ‘package:utils’:
#>
#> timestamp
data <- readRDS(“problem.rds”)
arrow::write_dataset(
data,
path = “~/problem”,
partitioning = c(“publication_year”, “page”),
format = “parquet”
)
#> Error: Invalid: Problem with column 3 (au_orcid): Invalid: Expecting a character vector
```
Created on 2024-03-21 with [reprex v2.1.0](https://reprex.tidyverse.org/)
[problem.rds.zip](https://github.com/apache/arrow/files/14696782/problem.rds.zip)

### Component(s)

R

Contributor guide

Open the contributing guide

Research direction

Start by loading the attached problem.rds in R and reproducing the failure with arrow::write_dataset(), using the issue's partitioning and parquet settings. Trace how the nested data.frame and the all-NA au_orcid column are converted, then verify that the same example saves successfully without the character-vector error.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.