apache / apache/arrow

[C++] Segfault when reading a Parquet file as a Dataset but not when read as an individual file

Open
#36,807 8 comments 0 reactions 0 assignees View on GitHub
Component: C++ Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

I'm using the Arrow R package version 12.0.1.1 and am getting segfault when trying to read a Parquet file. Here's the output with the debugger attached:

```
> library(fs)
library(arrow)
library(dplyr)
[New Thread 0x7ffff33ff640 (LWP 480350)]
[New Thread 0x7fffe99ff640 (LWP 480356)]
Some features are not enabled in this build of Arrow. Run `arrow_info()` for more information.

Attaching package: ‘arrow’

The following object is masked from ‘package:utils’:

timestamp

Attaching package: ‘dplyr’

The following objects are masked from ‘package:stats’:

filter, lag

The following objects are masked from ‘package:base’:

intersect, setdiff, setequal, union

> all_files <- dir_ls("/data/nyc-taxi", recurse=TRUE)
parquet_files <- all_files[endsWith(all_files, "parquet")]
> parquet_files[86]
/data/nyc-taxi/year=2016/month=10/part-0.parquet
> ds <- open_dataset(parquet_files[86]) %>% head(6) %>% collect()
[New Thread 0x7fffe9007640 (LWP 480358)]
[New Thread 0x7fffe8806640 (LWP 480359)]
[New Thread 0x7fffd7b7f640 (LWP 480360)]
[New Thread 0x7fffd6b7f640 (LWP 480361)]
[New Thread 0x7fffd637e640 (LWP 480362)]
[New Thread 0x7fffd5b7d640 (LWP 480363)]
[New Thread 0x7fffd537c640 (LWP 480364)]
[New Thread 0x7fffd4b7b640 (LWP 480365)]
[New Thread 0x7fffcd7ff640 (LWP 480366)]
[New Thread 0x7fffccffe640 (LWP 480367)]
[New Thread 0x7fffb3fff640 (LWP 480368)]
[New Thread 0x7fffb37fe640 (LWP 480369)]
[New Thread 0x7fffb2ffd640 (LWP 480370)]
> nrow(ds)
[1] 6
> parquet_files[87]
/data/nyc-taxi/year=2016/month=11/part-0.parquet
> ds <- open_dataset(parquet_files[87]) %>% head(6) %>% collect()
>
Thread 13 "R" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffccffe640 (LWP 480367)]
0x00007ffff00fbf38 in arrow::internal::Executor::Submit, const std::vector&, const std::vector&, arrow::internal::Executor*)::)>&, long unsigned int&, std::shared_ptr >(arrow::internal::TaskHints, arrow::StopToken, struct {...} &) (this=0x2e1c00000008, hints=..., stop_token=..., func=...) at /home/nic2/arrow/cpp/src/arrow/util/thread_pool.h:159
159 ARROW_RETURN_NOT_OK(SpawnReal(hints, std::move(task), std::move(stop_token),
```

If I read in the file via `read_parquet()`, I don't have a problem and it loads fine. Happy to supply the file if necessary, though wasn't sure it's possible/desirable to attach a 150Mb file to an issue ticket.

### Component(s)

C++

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.