[Python] StructArray.from_arrays does not work well with FixedSizeList-typed data
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
### Describe the bug, including details regarding any error messages, version, and platform.
Pyarrow's `StructArray.from_arrays` constructor will reject valid non-arrow data (for example a Python list-of-lists or Pandas Series) if the struct field specifies a fixed-length list.
Here's an example - first a bit of setup:
```py
import pyarrow as pa
import pandas as pd
dtype = pa.list_(pa.int64(), 2)
data_list = [[1, 2], [3, 4]]
data_pd_series = pd.Series(data_list)
data_array = pa.array(data_list, dtype)
```
Now, calling `pa.StructArray.from_arrays(foo, fields=[pa.field("x", dtype)])` should work regardless of whether `foo` is `data_list`, or `data_pd_series`, or `data_array`. But no:
```py
>>> sa = pa.StructArray.from_arrays([data_list], fields=[pa.field("x", dtype)])
Traceback (most recent call last):
File "", line 1, in
File "pyarrow/array.pxi", line 2886, in pyarrow.lib.StructArray.from_arrays
File "pyarrow/array.pxi", line 1596, in pyarrow.lib.Array.validate
File "pyarrow/error.pxi", line 100, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Struct child array #0 does not match type field: list vs fixed_size_list[2]
>>> sa = pa.StructArray.from_arrays([data_pd_series], fields=[pa.field("x", dtype)])
Traceback (most recent call last):
File "", line 1, in
File "pyarrow/array.pxi", line 2886, in pyarrow.lib.StructArray.from_arrays
File "pyarrow/array.pxi", line 1596, in pyarrow.lib.Array.validate
File "pyarrow/error.pxi", line 100, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Struct child array #0 does not match type field: list vs fixed_size_list[2]
>>>
>>> sa = pa.StructArray.from_arrays([data_array], fields=[pa.field("x", dtype)])
```
It appears that `pa.StructArray.from_arrays` casts non-pyarrow types, and _then_ validates against the schema without attempting another cast, while `pa.array` works differently somehow.
### Component(s)
Python
Contributor guide
Assessment
This issue has not been assessed yet.