apache / apache/arrow-nanoarrow
[R] Convert arrow dictionary to R factor via `as.data.frame.nanoarrow_array_stream()`?
- Dominant language
- C
- Stars
- 249
- Forks
- 68
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 11
Description
Maybe related to #220
I noticed that if we convert nanoarrow_array_stream to data.frame, the dictionary becomes a character.
``` r
stream <- data.frame(
x = as.factor(letters[1:5]),
y = as.factor(1:5)
) |>
nanoarrow::as_nanoarrow_array_stream()
stream
#> , y: dictionary(int32)>>
#> $ get_schema:function ()
#> $ get_next :function (schema = x$get_schema(), validate = TRUE)
#> $ release :function ()
```
``` r
stream |>
tibble::as_tibble()
#> # A tibble: 5 × 2
#> x y
#>
#> 1 a 1
#> 2 b 2
#> 3 c 3
#> 4 d 4
#> 5 e 5
```
Created on 2024-06-09 with [reprex v2.1.0](https://reprex.tidyverse.org)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the conversion shown in the issue through as.data.frame.nanoarrow_array_stream() and compare the resulting columns with the original factors. Trace the dictionary handling in that entry point; done means dictionary-encoded columns are returned as R factors rather than characters, with coverage for both example columns.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100