apache / apache/arrow

[Python] MapArray pandas round trip is broken

Open
#31,965 3 comments 0 reactions 0 assignees View on GitHub
Component: Python Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 18h
Merged PRs (30d)
91

Description

pyarrow.MapArray when converted to pandas cannot be successfully converted back.

The following snipper does not work:

 

```
import pyarrow as pa

data = [[('x', 1), ('y', 0)], [('a', 2), ('b', 45)]]
ty = pa.map_(pa.string(), pa.int64())
map_col = pa.array(data, type=ty)
pa.MapArray.from_pandas(map_col.to_pandas())
```
`Uncaught exception: ArrowTypeError: Expected bytes, got a 'int' object (java.lang.RuntimeException)`

**Reporter**: [Robbie Gruener](https://issues.apache.org/jira/browse/ARROW-16611) / @rgruener

**Note**: *This issue was originally created as [ARROW-16611](https://issues.apache.org/jira/browse/ARROW-16611). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Research direction

Start by running the Python reproduction using MapArray.from_pandas and map_col.to_pandas to confirm the round-trip failure. Trace those conversion entry points and verify that the same map data converts to pandas and back without the reported ArrowTypeError.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, python
Domain
data-engineering
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.