apache / apache/arrow

[Python] Convert multidimensional arrays into FixedShapeTensorArrays automatically in `pyarrow.array`

Open
#35,647 5 comments 3 reactions 0 assignees View on GitHub
Component: Python Status: needs champion Type: enhancement
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the enhancement requested

Now that there is a canonical `fixed_shape_tensor` type, I was wondering whether there'd be any appetite for making `pyarrow.array` automatically convert numpy arrays with >1 dimension into `FixedShapeTensorArray`s.

You can do this manually by using `pa.FixedShapeTensorArray.from_numpy_ndarray`, but it seems more convenient for `pyarrow.array` to automatically support this.

If there's appetite for this, I think I'd be able to contribute a patch.

### Component(s)

Python

Contributor guide

Open the contributing guide

Research direction

Start at pyarrow.array and compare its handling of multidimensional NumPy arrays with FixedShapeTensorArray.from_numpy_ndarray. Done means multidimensional NumPy arrays are automatically converted to FixedShapeTensorArrays while existing pyarrow.array behavior remains intact; the issue does not name specific files or tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.