[cpp][parquet] Segmentation fault when open parquet file
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 91
Description
Hi,
I am new to Arrow and Parquet.
I installed Arrow 9.0.0 and Parquet following the guide in this repo.
When I tried to open the parquet file using follow CPP code (same as the example code in docs), I got a segmentation fault showing below when calling `parquet::arrow::OpenFile`
```CPP
arrow::Status st;
arrow::MemoryPool *pool = arrow::default_memory_pool();
arrow::fs::LocalFileSystem file_system;
std::shared_ptr input =
file_system.OpenInputFile("/root/path/to/file.parquet").ValueOrDie();
// Open Parquet file reader
std::unique_ptr arrow_reader;
st = parquet::arrow::OpenFile(input, pool, &arrow_reader);
```
I backtracked the stack info in gdb:
```bash
(gdb) bt
#0 0x00007ffff5ed1f74 in arrow::Schema::num_fields() const () from /lib64/libarrow.so.800
#1 0x00007ffff7788a41 in parquet::arrow::SchemaManifest::Make(parquet::SchemaDescriptor const*, std::shared_ptr const&, parquet::ArrowReaderProperties const&, parquet::arrow::SchemaManifest*) () from /lib64/libparquet.so.800
#2 0x00007ffff7755395 in parquet::arrow::FileReader::Make(arrow::MemoryPool*, std::unique_ptr >, parquet::ArrowReaderProperties const&, std::unique_ptr >*) () from /lib64/libparquet.so.800
#3 0x00007ffff7755601 in parquet::arrow::FileReaderBuilder::Build(std::unique_ptr >*) () from /lib64/libparquet.so.800
#4 0x00007ffff77563b9 in parquet::arrow::OpenFile(std::shared_ptr, arrow::MemoryPool*, std::unique_ptr >*) () from /lib64/libparquet.so.800
#5 0x000000000040a4b6 in main (argc=1, argv=0x7fffffffdba8) at /root/mini-sim/mdl/src/main.cpp:65
(gdb)
```
But I can open and read the same file using pandas in python
```python
pandas.read_parquet("/root/path/to/file.parquet")
```
What did I miss?
I really appreciate any help you can provide.
Contributor guide
Research direction
Start with the C++ reproducer at parquet::arrow::OpenFile and the gdb stack through parquet::arrow::SchemaManifest::Make and arrow::Schema::num_fields(). Check the Arrow and Parquet library versions involved, including the loaded libarrow.so.800 and libparquet.so.800, then reproduce with the reported parquet file. Done means identifying the cause of the segmentation fault and confirming that opening the file no longer crashes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100