apache / apache/arrow

[R] crash when reading with duckdb

Open
#40,593 5 comments 0 reactions 0 assignees View on GitHub
Component: R Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

Using arrow 15 in R through dplyr, I got a crash (R session is crashed) when reading a dataset with to_duckdb(). On my dataset, pretty large composed of some parquet files -- I can send a dropbox link if needed, there are 2 columns that provoque the crash. There are encoded as factors (dictionary) and I suspect that different dictionaries for the different parquet are the cause of the crash.
so `open_dataset(mydts) |> select(field1) |> to_duckdb() |> collect()` crash everything
`open_dataset(mydts) |> select(field1) |> mutate(field1 = as.character(field1) |> to_duckdb() |> collect()` works
and
`open_dataset(mydts) |> select(field1) |> collect()` works too.

I tried to rewrite the dataset loading it first (without duckdb) and then using write_dataset with no succes.

using arrow 15.0.1 and duckdb 0.10

### Component(s)

R

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.