apache / apache/arrow

Store a `extension<arrow.fixed_size_tensor>` in a `pyarrow.Table` and retrieve it back reshaped

Open
#36,890 4 comments 0 reactions 0 assignees View on GitHub
Component: Python Type: usage
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the usage question you have. Please include as many useful details as possible.

I'm following the example from the official [docs](https://arrow.apache.org/docs/python/extending_types.html#fixed-size-tensor) on how to create a fixed size tensor and store it in a `pyarrow.Table`:

```python
data = [
pa.array([1, 2, 3]),
pa.array(['foo', 'bar', None]),
pa.array([True, None, True]),
tensor_array,
tensor_array_2
]
my_schema = pa.schema([('f0', pa.int8()),
('f1', pa.string()),
('f2', pa.bool_()),
('tensors_int', tensor_type),
('tensors_float', tensor_type_2)])
table = pa.Table.from_arrays(data, schema=my_schema)
table
```

While it's possible to do:

```python
numpy_tensor = tensor_array_2.to_numpy_ndarray()
```

I cannot achieve the same if I retrieve `tensor_array_2` from the table like this:

```python
table.column("tensors_float").to_numpy_ndarray()
```

I get the following error:

```
Traceback (most recent call last):
File "", line 1, in
AttributeError: 'pyarrow.lib.ChunkedArray' object has no attribute 'to_numpy_ndarray'
```

### Component(s)

Python

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.