apache / apache/arrow

[Python] Add a pyarrow.Table.aggregate function to compute aggregates against the whole table

Open
#14,896 3 comments 0 reactions 0 assignees View on GitHub
Component: Python Type: enhancement
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 18h
Merged PRs (30d)
91

Description

### Describe the enhancement requested

The implementation would be almost identical to pyarrow.TableGroupBy.aggregate except the keys list would be empty. Currently this does not appear to be possible using an empty array of keys:

```
>>> tab.group_by([]).aggregate([("x", "sum")])
Traceback (most recent call last):
File "", line 1, in
File "pyarrow/table.pxi", line 5325, in pyarrow.lib.TableGroupBy.aggregate
File "pyarrow/_compute.pyx", line 2145, in pyarrow._compute._group_by
File "pyarrow/error.pxi", line 144, in pyarrow.lib.pyarrow_internal_check_status
File "pyarrow/error.pxi", line 100, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Cannot infer ExecBatch length without at least one value
```

Even if it were possible, I don't think it would be obvious to obtain the result in this way.

### Component(s)

Python

Contributor guide

Open the contributing guide

Research direction

Start with pyarrow.TableGroupBy.aggregate in pyarrow/table.pxi and the _group_by implementation in pyarrow/_compute.pyx. Compare the requested Table.aggregate behavior with the empty-key example, and consider the work complete when whole-table aggregation no longer raises and produces the expected aggregate result.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.