apache / apache/arrow

[R] "open_dataset(f) %>% filter(id %in% myvec) %>% collect" causes CPP11::unwind_execption, crashed R

Open
#33,487 3 comments 0 reactions 0 assignees View on GitHub
Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

This is running on a windows environment, arrow 10.0.0 (see arrow_info() below). data size is large maybe 

I issued two calls

```

ft <- path_to_dataset1
fa <- path_to_dataset2

#1)

tic()
d2 <- ft %>% open_dataset %>% filter( pis %in% mypis ) %>% collect
toc()
927.11 sec elapsed

#returned a dataset with 44 obs, 38 columns, took abnormal time, 16min

#1)

tic()
d3 <- fa %>% open_dataset %>% filter( pis %in% mypis ) %>% collect
terminate called after throwing an instance of 'cpp11::unwind_exception'

```

Then I got an error that craspad_hendler.exe stopped working. And R becomes frozen, after a while R crashed too.

![image-2022-11-11-14-59-30-132.png](https://issues.apache.org/jira/secure/attachment/13052122/image-2022-11-11-14-59-30-132.png)

 

arrow_info()
Arrow package version: 10.0.0

Capabilities:
               
dataset    TRUE
substrait FALSE
parquet    TRUE
json       TRUE
s3         TRUE
gcs        TRUE
utf8proc   TRUE
re2        TRUE
snappy     TRUE
gzip       TRUE
brotli     TRUE
zstd       TRUE
lz4        TRUE
lz4_frame  TRUE
lzo       FALSE
bz2        TRUE
jemalloc  FALSE
mimalloc   TRUE

Arrow options():
                       
arrow.use_threads FALSE

Memory:
                  
Allocator mimalloc
Current    0 bytes
Max        0 bytes

Runtime:
                        
SIMD Level          avx2
Detected SIMD Level avx2

Build:
                                                             
C++ Library Version                                    10.0.0
C++ Compiler                                              GNU
C++ Compiler Version                                   10.3.0
Git ID               aa7118b6e5f49b354fa8a93d9cf363c9ebe9a3f0

 

 

 

**Reporter**: [Lucas Mation](https://issues.apache.org/jira/browse/ARROW-18314) / @lucasmation
#### Original Issue Attachments:
- [image-2022-11-11-14-55-36-430.png](https://issues.apache.org/jira/secure/attachment/13052123/image-2022-11-11-14-55-36-430.png)
- [image-2022-11-11-14-59-30-132.png](https://issues.apache.org/jira/secure/attachment/13052122/image-2022-11-11-14-59-30-132.png)

**Note**: *This issue was originally created as [ARROW-18314](https://issues.apache.org/jira/browse/ARROW-18314). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Research direction

Start with the R pipeline using open_dataset, filter(pis %in% mypis), and collect on Windows, then review the supplied arrow_info() output for Arrow 10.0.0 and the cpp11::unwind_exception crash. Compare the two dataset calls and establish a reproducible case; done means the query no longer freezes or crashes R and its result remains usable.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, r
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.