apache / apache/arrow

`fixed_size_list<T>[0]` parquet round-trips fail: writes fine, read-back raises `ArrowInvalid`

Open
#49,683 4 comments 0 reactions 0 assignees View on GitHub
Component: Parquet Component: Python Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

A `fixed_size_list` array with `list_size=0` can be written to Parquet successfully but fails to read back with
```
ArrowInvalid: Expected all lists to be of size=0 but index 1 had size=1
```

Note also that the high-level constructor `pa.FixedSizeListArray.from_arrays` explicitly rejects `list_size=0` with `"list_size needs to be a strict positive integer"`, but the type can still be constructed via `Array.from_buffers`, and `pq.write_table` accepts it without complaint. The API is inconsistent: either the type should be rejected everywhere, or it should round-trip correctly.

```
inner = pa.array([], type=pa.float32()) # 0 child values
fsl = pa.FixedSizeListArray.from_arrays(inner, list_size=0)
```
raises
```
ArrowInvalid: list_size needs to be a strict positive integer
```

### Reproducer
```python
import pyarrow as pa
import pyarrow.parquet as pq

fsl = pa.Array.from_buffers(
pa.list_(pa.float32(), 0),
length=50,
buffers=[None],
children=[pa.array([], type=pa.float32())],
)
table = pa.table({"x": fsl})

pq.write_table(table, "tmp.parquet") # OK
pq.read_table("tmp.parquet") # ArrowInvalid
```

### Component(s)

Parquet, Python

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.