apache / apache/arrow

[cpp][parquet] Segmentation fault when open parquet file

Open
#13,801 9 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 18h
Merged PRs (30d)
91

Description

Hi,
I am new to Arrow and Parquet.
I installed Arrow 9.0.0 and Parquet following the guide in this repo.

When I tried to open the parquet file using follow CPP code (same as the example code in docs), I got a segmentation fault showing below when calling `parquet::arrow::OpenFile`

```CPP
arrow::Status st;
arrow::MemoryPool *pool = arrow::default_memory_pool();
arrow::fs::LocalFileSystem file_system;
std::shared_ptr input =
file_system.OpenInputFile("/root/path/to/file.parquet").ValueOrDie();

// Open Parquet file reader
std::unique_ptr arrow_reader;
st = parquet::arrow::OpenFile(input, pool, &arrow_reader);
```

I backtracked the stack info in gdb:

```bash
(gdb) bt
#0 0x00007ffff5ed1f74 in arrow::Schema::num_fields() const () from /lib64/libarrow.so.800
#1 0x00007ffff7788a41 in parquet::arrow::SchemaManifest::Make(parquet::SchemaDescriptor const*, std::shared_ptr const&, parquet::ArrowReaderProperties const&, parquet::arrow::SchemaManifest*) () from /lib64/libparquet.so.800
#2 0x00007ffff7755395 in parquet::arrow::FileReader::Make(arrow::MemoryPool*, std::unique_ptr >, parquet::ArrowReaderProperties const&, std::unique_ptr >*) () from /lib64/libparquet.so.800
#3 0x00007ffff7755601 in parquet::arrow::FileReaderBuilder::Build(std::unique_ptr >*) () from /lib64/libparquet.so.800
#4 0x00007ffff77563b9 in parquet::arrow::OpenFile(std::shared_ptr, arrow::MemoryPool*, std::unique_ptr >*) () from /lib64/libparquet.so.800
#5 0x000000000040a4b6 in main (argc=1, argv=0x7fffffffdba8) at /root/mini-sim/mdl/src/main.cpp:65
(gdb)
```

But I can open and read the same file using pandas in python

```python
pandas.read_parquet("/root/path/to/file.parquet")
```

What did I miss?
I really appreciate any help you can provide.

Contributor guide

Open the contributing guide

Research direction

Start with the C++ reproducer at parquet::arrow::OpenFile and the gdb stack through parquet::arrow::SchemaManifest::Make and arrow::Schema::num_fields(). Check the Arrow and Parquet library versions involved, including the loaded libarrow.so.800 and libparquet.so.800, then reproduce with the reported parquet file. Done means identifying the cause of the segmentation fault and confirming that opening the file no longer crashes.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.