[c++][parquet][8.0.0] The coredump appears when constructed parquetfileReader
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 91
Description
arrow version 8.0.0 linux
Not necessarily, It's been running for a while。
```
do {
auto access_file_result =
s3fs_->OpenInputFile(dataset()->parquet_files_[file_index]);
if (!access_file_result.ok()) {
res =
errors::InvalidArgument(access_file_result.status().ToString());
break;
}
auto access_file = access_file_result.ValueOrDie();
parquet::ArrowReaderProperties properties;
properties.set_use_threads(true);
properties.set_pre_buffer(true);
parquet::ReaderProperties parquet_properties =
parquet::default_reader_properties();
std::shared_ptr builder =
std::make_shared();
builder->Open(access_file, parquet_properties);
std::unique_ptr reader;
builder->properties(properties)->Build(&reader);
....
}while(0);
```
call stack
```
(gdb) bt
#0 0x00007f34f9dd6704 in parquet::ParquetFileReader::metadata() const ()
from /usr/local/lib/python3.8/dist-packages/tensorflow_io/python/ops/libtensorflow_io.so
#1 0x00007f34f9f572b9 in parquet::arrow::FileReader::Make(arrow::MemoryPool*, std::unique_ptr >, parquet::ArrowReaderProperties const&, std::unique_ptr >*) () from /usr/local/lib/python3.8/dist-packages/tensorflow_io/python/ops/libtensorflow_io.so
#2 0x00007f34f9f573ec in parquet::arrow::FileReaderBuilder::Build(std::unique_ptr >*) () from /usr/local/lib/python3.8/dist-packages/tensorflow_io/python/ops/libtensorflow_io.so
#3 0x00007f34f9cdd611 in tensorflow::data::ArrowS3DatasetOp::Dataset::Iterator::ReadFile(int, bool) ()
from /usr/local/lib/python3.8/dist-packages/tensorflow_io/python/ops/libtensorflow_io.so
#4 0x00007f34f9cded3c in tensorflow::data::ArrowS3DatasetOp::Dataset::Iterator::SetupStreamsLocked(tensorflow::Env*) ()
from /usr/local/lib/python3.8/dist-packages/tensorflow_io/python/ops/libtensorflow_io.so
#5 0x00007f34f9cff311 in tensorflow::data::ArrowDatasetBase::ArrowBaseIterator::GetNextInternal(tensorflow::data::IteratorContext*, std::vector >*, bool*) ()
from /usr/local/lib/python3.8/dist-packages/tensorflow_io/python/ops/libtensorflow_io.so
#6 0x00007f3e57badaa4 in tensorflow::data::DatasetBaseIterator::GetNext(tensorflow::data::IteratorContext*, std::vector >*, bool*) ()
from /usr/local/lib/python3.8/dist-packages/tensorflow/python/../libtensorflow_framework.so.2
#7 0x00007f3e720938f6 in tensorflow::data::ParallelMapDatasetOp::Dataset::Iterator::CallFunction(std::shared_ptr const&, std::shared_ptr const&) ()
from /usr/local/lib/python3.8/dist-packages/tensorflow/python/_pywrap_tensorflow_internal.so
#8 0x00007f3e72099022 in tensorflow::data::ParallelMapDatasetOp::Dataset::Iterator::RunnerThread(std::shared_ptr const&) () from /usr/local/lib/python3.8/dist-packages/tensorflow/python/_pywrap_tensorflow_internal.so
#9 0x00007f3e70c6e145 in tensorflow::data::(anonymous namespace)::WorkQueueFunc(std::function const&, std::shared_ptr) () from /usr/local/lib/python3.8/dist-packages/tensorflow/python/_pywrap_tensorflow_internal.so
#10 0x00007f3e70c6ef5d in std::_Function_handler, std::shared_ptr))(std::function const&, std::shared_ptr)> >::_M_invoke(std::_Any_data const&) ()
from /usr/local/lib/python3.8/dist-packages/tensorflow/python/_pywrap_tensorflow_internal.so
#11 0x00007f3e58356791 in tensorflow::UnboundedWorkQueue::PooledThreadFunc() ()
from /usr/local/lib/python3.8/dist-packages/tensorflow/python/../libtensorflow_framework.so.2
#12 0x00007f3e5835d5d8 in tensorflow::(anonymous namespace)::PThread::ThreadFn(void*) ()
from /usr/local/lib/python3.8/dist-packages/tensorflow/python/../libtensorflow_framework.so.2
#13 0x00007f3f3f931609 in start_thread (arg=) at pthread_create.c:477
#14 0x00007f3f3fa6b133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
```
Are there any tips to troubleshoot this error?
### Component(s)
C++
Contributor guide
Research direction
Start with the provided OpenInputFile, FileReaderBuilder::Open, properties, and Build sequence, then inspect the supplied gdb backtrace through ParquetFileReader::metadata(). Reproduce the coredump on Arrow 8.0.0 on Linux if possible; done means identifying a reliable trigger or providing a verified fix and regression coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100