apache / apache/arrow

[Python] Reading and filtering a parquet file from s3 causes 139 exit

Open
#39,162 6 comments 0 reactions 0 assignees View on GitHub
Component: Parquet Component: Python Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

Reading a parquet file from s3 using `pyarrow.parquet.read_table` seems to intermittently cause a exit code 139 error when running code through a docker container with the message `terminate called without an active exception` when you add filters that filter out all of the data.
```
import fsspec
import pyarrow.parquet as pq
fs, path = fsspec.core_url_to_fs(s3_file_path)

dataset = pq.read_table(path, filesystem=fs, filters= [[('date', '<', '2000-01-01')]])
```

### Environment
Python 3.8
fsspec==2023.12.0
pyarrow==14.0.1 - * using pyarrow==3.0.1 seems to work

### Component(s)

Parquet, Python

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.