apache / apache/arrow

[Python] `FixedShapedTensor` `to_pandas_dtype` is returning `NotImplementedError`

Open
#49,907 1 comment 0 reactions 0 assignees View on GitHub
Component: Python Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

Repro Script:
```
import pyarrow as pa

print(f"pyarrow {pa.__version__}")

t = pa.fixed_shape_tensor(pa.float32(), [2, 2])
print(f"type: {t!r}")

print("calling t.to_pandas_dtype() ...")
dtype = t.to_pandas_dtype()
print(f"result: {dtype!r}")
```

Output:
```
pyarrow 23.0.0
type: FixedShapeTensorType(extension)
calling t.to_pandas_dtype() ...
---------------------------------------------------------------------------
NotImplementedError Traceback (most recent call last)
Cell In[2], line 9
6 print(f"type: {t!r}")
8 print("calling t.to_pandas_dtype() ...")
----> 9 dtype = t.to_pandas_dtype()
10 print(f"result: {dtype!r}")
NotImplementedError: extension
```

Seems to contradict what's available in the docs: https://arrow.apache.org/docs/python/generated/pyarrow.FixedShapeTensorType.html#pyarrow.FixedShapeTensorType

For context, this method needs to be implemented in order to use `table.to_pandas(split_blocks=True)` for per-column extension blocks to reduce memory pressure.

### Component(s)

Python

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.