apache / apache/arrow

[Python] Support pandas dtype_backend=pyarrow in to_pandas

Open
#35,802 7 comments 2 reactions 0 assignees View on GitHub
Component: Python Type: enhancement
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 18h
Merged PRs (30d)
91

Description

### Describe the enhancement requested

As pandas 2.0 start supporting pyarrow as engine and dtype.
If we can support it in [pyarrow.Table.to_pandas()](https://arrow.apache.org/docs/python/generated/pyarrow.Table.html#pyarrow.Table.to_pandas) will be really helpful and efficient.

### Component(s)

Python

Contributor guide

Open the contributing guide

Research direction

Start with the Python pyarrow.Table.to_pandas() entry point mentioned in the issue and review pandas 2.0's dtype_backend documentation. Determine the expected behavior for dtype_backend=pyarrow and identify the relevant conversion tests before changing anything. Done means pandas conversions accept the pyarrow backend and produce correct results.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, python
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.