apache / apache/arrow

[Python] Several APIs segfault when required Arrow object arguments are `None`

Open
#51,043 1 comment 0 reactions 0 assignees View on GitHub
Component: Python Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 18h
Merged PRs (30d)
91

Description

### Summary

Several PyArrow APIs whose signatures require Arrow extension objects reject invalid types with `TypeError`, but accept `None` and then terminate the interpreter.
I would expect `None` to be rejected with `TypeError`, like the other invalid types, rather than causing a process crash.
I found these cases while fuzzing Python C extension modules.

### Versions

PyArrow 25.0.1, CPython 3.12.3, Debian 12 x86_64, glibc 2.36

### Reproducer

Run each command independently in a fresh process.

```console
python -c 'import pyarrow.parquet as pq; pq.SortingColumn.from_ordering(None, [])'
python -c 'import pyarrow as pa; import pyarrow.dataset as ds; ds.FileSystemDataset([None], pa.schema([]), ds.ParquetFileFormat())'
python -c 'import pyarrow as pa; dictionary=pa.array([], type=pa.string()); buffers=[None, pa.py_buffer(b"")]; pa.DictionaryArray.from_buffers(None, 0, buffers, dictionary)'
```

```text
SortingColumn.from_ordering SIGSEGV (signal 11)
FileSystemDataset SIGSEGV (signal 11)
DictionaryArray.from_buffers SIGSEGV (signal 11)
```

### ASan/UBSan result

I built PyArrow 25.0.0 from source with Clang 18 using ASan and UBSan instrumentation.
The first sanitizer failures were:

| API | First native failure |
| --- | --- |
| `SortingColumn.from_ordering()` | member call on a null `arrow::Schema` in `parquet::arrow::ToParquetSchema()` at `cpp/src/parquet/arrow/schema.cc:1233` |
| `FileSystemDataset()` | member access through a null `FileFragment` extension pointer at `build/_dataset.cpp:23011` |
| `DictionaryArray.from_buffers()` | member call on a null `arrow::DataType` in `arrow::ArrayData::Make()` at `cpp/src/arrow/array/data.cc:118` |

ASan subsequently reports zero-page reads on the corresponding native paths and aborts each process with exit code 134.

### Component(s)

Python

Contributor guide

Open the contributing guide

Research direction

Run each Python reproducer in a fresh process, then inspect the native paths named in the sanitizer table: cpp/src/parquet/arrow/schema.cc:1233, build/_dataset.cpp:23011, and cpp/src/arrow/array/data.cc:118. Trace how None reaches each required Arrow argument and review nearby Python binding tests. Done means all three calls raise TypeError instead of terminating the interpreter, with regression coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
api, data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.