apache / apache/arrow

[Python] Recognize pandas arrays in the array() constructor for the from_pandas argument

Open
#35,721 0 comments 0 reactions 0 assignees View on GitHub
Component: Python Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 18h
Merged PRs (30d)
91

Description

The `from_pandas` argument in the `array(..)` constructor determines whether NaN-like values are converted to null, or preserved as NaN. If not specified by the user, we will set the keyword automatically when detecting the input as a "pandas object". Currently we only check for Series and Index, but we should also check for ExtensionArrays:

```
>>> pa.array([1.2, np.nan])

[
1.2,
nan
]
>>> pa.array([1.2, np.nan], from_pandas=True)

[
1.2,
null
]
>>> pa.array(pd.Series([1.2, np.nan]))

[
1.2,
null
]
>>> pa.array(pd.array([1.2, np.nan], dtype=float))

[
1.2,
nan
]
```

Contributor guide

Open the contributing guide

Research direction

Start at the Python array() constructor and the logic that detects pandas objects for the from_pandas argument. Reproduce the examples with a pandas ExtensionArray, then verify that automatic detection converts NaN-like values to null while preserving explicit behavior for other inputs.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.