apache / apache/arrow

Bug in pyarrow.from_pandas() when input has MultiIndex index columns having non-string names

Open
#38,983 3 comments 0 reactions 0 assignees View on GitHub
Component: Python Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

When the input dataframe has MultiIndex index columns having non-string names, the output is not as expected:
```
import pyarrow as pa
print(f"version: {pa.__version__}")

import pandas as pd
index = pd.MultiIndex.from_tuples([(10,20), (30,40), (50,60), (70,80)], names = [1,2])
df = pd.DataFrame({"a": [1,2,3,4], "b": [5,6,7,8]}, index=index)
print(df)
print(pa.Table.from_pandas(df))
```

Outputs:
```
version: 14.0.1
sys:1: UserWarning: The DataFrame has non-str index name `[1, 1]` which will be converted to string and not roundtrip correctly.
pyarrow.Table
a: int64
b: int64
1: int64
1: int64
----
a: [[1,2,3,4]]
b: [[5,6,7,8]]
1: [[10,30,50,70]]
1: [[10,30,50,70]]
```

### Component(s)

Python

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.