[Python]: Possible to evaluate `pyarrow.compute.Expression` without filter?
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 91
Description
### Describe the usage question you have. Please include as many useful details as possible.
I'm unsure whether this should be categorized as "usage" or "enhancement".
I've read through the [high-level compute functions doc](https://arrow.apache.org/docs/python/compute.html), as well as the [compute API doc page](https://arrow.apache.org/docs/python/api/compute.html), and also looked through `compute.py` and `_compute.pyx`.
Is there a pyarrow compute API for evaluating an expression against a table or array, receiving _indices_ as output? All the docs seem to use `table.filter`, whereas I'm looking for something like [`numpy.where`](https://numpy.org/doc/stable/reference/generated/numpy.where.html), where I can use `where` and then `table.take` in two different steps instead of one. Presumably `filter` is already doing a "where" then "take" under the hood?
```py
import pyarrow.compute as pc
import pyarrow as pa
table = pa.Table.from_arrays([pa.array([1, 2, 3, 4])], names=["a"])
expr = pc.field('a') == 2
# does this exist?
table.evaluate(expr)
# or maybe
table.where(expr)
# Expected output BooleanArray:
# pa.array([False, True, False, False])
```
### Component(s)
Python
Contributor guide
Research direction
Start with the high-level compute documentation, the compute API documentation, compute.py, and _compute.pyx, as identified in the issue. Check whether an expression can produce a BooleanArray or indices independently of table.filter, and compare the requested behavior with the shown table and expression. Done means establishing the supported API or documenting the missing capability and its expected output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100