apache / apache/arrow

[Python] Enhance pyarrow.compute to Support Broadcasting Similar to numpy

Open
#44,544 13 comments 0 reactions 0 assignees View on GitHub
Component: Python Type: enhancement
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the enhancement requested

`numpy` supports broadcasting, allowing operations between arrays of different lengths (e.g., multi-element and single-element arrays). It would be beneficial if pyarrow could implement similar behavior.

Example:

```python
import numpy as np
import pyarrow as pa

# Works in numpy
np.array([1, 2, 3]) + np.array([4])

# Also works in pyarrow with a scalar
pa.compute.add(pa.array([1, 2, 3]), 4)

# Doesn't work in pyarrow with an array - raises
# ArrowInvalid: Array arguments must all be the same length
pa.compute.add(pa.array([1, 2, 3]), pa.array([4]))
```

This issue arose from a discussion in a related Pandas issue: https://github.com/pandas-dev/pandas/issues/60073.

### Component(s)

Python

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.