[Python] Cannot use sparse_union/dense_union types with pa.Table.from_pylist()
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
### Describe the bug, including details regarding any error messages, version, and platform.
An exception is raised when attempting to use a sparse or dense union with `pa.Table.from_pylist()`.
**Steps to Reproduce**:
Run the following Python code:
```python
import pyarrow as pa
schema = pa.schema([
pa.field("data",
pa.sparse_union([
pa.field("age", pa.int32()),
pa.field("weight", pa.int32())
])
)
])
data = [
{"data": {"age": 20}},
]
table = pa.Table.from_pylist(data, schema=schema)
```
**Expected Behavior:**
The code should create a `pa.Table` using the provided schema and data.
**Actual Behavior**
The following exception is raised:
```
pyarrow.lib.ArrowNotImplementedError: sparse_union
```
With a traceback of:
```
Traceback (most recent call last):
File "/Users/rusty/Development/flight-cloud/src/flight_cloud/example-union.py", line 7, in
table = pa.Table.from_pylist(data, schema=schema)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "pyarrow/table.pxi", line 1984, in pyarrow.lib._Tabular.from_pylist
File "pyarrow/table.pxi", line 6230, in pyarrow.lib._from_pylist
File "pyarrow/table.pxi", line 4758, in pyarrow.lib.Table.from_arrays
File "pyarrow/table.pxi", line 1562, in pyarrow.lib._sanitize_arrays
File "pyarrow/array.pxi", line 398, in pyarrow.lib.asarray
File "pyarrow/array.pxi", line 368, in pyarrow.lib.array
File "pyarrow/array.pxi", line 42, in pyarrow.lib._sequence_to_array
File "pyarrow/error.pxi", line 155, in pyarrow.lib.pyarrow_internal_check_status
File "pyarrow/error.pxi", line 92, in pyarrow.lib.check_status
pyarrow.lib.ArrowNotImplementedError: sparse_union
```
This occurs both when using `pa.sparse_union` and `pa.dense_union`.
**PyArrow version: 18.0.0.dev**
### Component(s)
Python
Contributor guide
Assessment
This issue has not been assessed yet.