apache / apache/arrow

Custom data types in arrow array

Open
#38,559 5 comments 0 reactions 0 assignees View on GitHub
Component: Python Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

Hello,

Given a simple class like this:

```python
import pyarrow as pa

class Point:
def __init__(self, iterable):
self._data = iterable
@property
def data(self) -> dict:
return self._data

pt = Point({"x" : 1,"y": 2,})
dataset = [pt]

pa.chunked_array([dataset])
```

How can I create a custom data type so it can be stored in the pyarrow Array or ChunkArray?

When I try to add a list of these item types, I get the following:

```pyarrow.lib.ArrowInvalid: Could not convert with type Point: did not recognize Python value type when inferring an Arrow data type```

Is there a way to allow for these custom objects to exist in an arrow array?

### Component(s)

Python

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.