apache / apache/arrow

[Python] Support schema-like objects through the Arrow PyCapule protocol as type or schema argument in the constructors

Open
#41,065 0 comments 1 reaction 0 assignees View on GitHub
Component: Python
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

See https://github.com/apache/arrow/pull/40818#issuecomment-2042199016 for context.

Our constructor functions like `pa.array(..)`, `pa.chunked_array(..)`, `pa.record_batch(..)` and `pa.table(..)` accept a `type` or `schema` argument.
Those constructors can all be used to consume data that support the Arrow PyCapsule protocol, and to convert those objects to the equivalent pyarrow object. For those cases, it would make sense to also accept the `type` or `schema` keyword argument as any object supporting the capsule protocol for schemas, i.e. having a `__arro_c_schema__`, while right now we require an actual pyarrow object.

One question is whether we would always support schema-like objects with the ArrowSchema protocol as input wherever specifying a type or schema, or whether we _only_ allow this when actually consuming data through the protocol (i.e. if the data object itself has the `__arrow_c_array/stream__`)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.