apache / apache/arrow-go

[Compute] The registry holds no aggregate functions (no sum, mean, min_max, count, hash_*), and the package documentation does not say so

Open
#1,296 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Assembly
Stars
404
Forks
145
Avg merge
2d 4h
Merged PRs (30d)
87

Description

**Describe the enhancement requested**

`compute.GetFunctionRegistry()` in v18.7.0 registers 85 functions, all scalar (80) or vector (5).
`FuncScalarAgg` and `FuncHashAgg` exist as kinds, but no function of either kind is registered: `sum`,
`mean`, `min`, `max`, `min_max`, `count`, `count_distinct`, `any`, `all`, `variance`, `stddev`,
`hash_sum` and `hash_count` are all absent. The package documentation describes "an Acero-like arrow
compute engine" and marks the package experimental, but does not say that aggregation is not
implemented, so a reader coming from pyarrow.compute or the C++ function list discovers it by a
failed lookup.

Go 1.27.1, macOS 26.6 (arm64), arrow-go v18.7.0. On v18.8.0-rc1 the registry holds 89 functions (76
scalar, 8 vector, 5 meta), still none of aggregate kind.

```go
reg := compute.GetFunctionRegistry()
names := reg.GetFunctionNames()
kinds := map[string]int{}
for _, n := range names {
fn, _ := reg.GetFunction(n)
kinds[fn.Kind().String()]++
}
fmt.Println(len(names), kinds)
for _, n := range []string{"sum", "mean", "min_max", "count", "hash_sum"} {
_, ok := reg.GetFunction(n)
fmt.Println(n, ok)
}
```

Output:

```
85 map[Scalar:80 Vector:5]
sum false
mean false
min_max false
count false
hash_sum false
```

**What would help**

Either of two things, and ideally both: a sentence in the `compute` package documentation listing
what is and is not implemented (scalar and vector functions yes; scalar aggregates and hash
aggregates not yet), and the aggregates themselves, starting with `sum`, `min_max`, `count` and
`mean` over the primitive types. Context: a binding that uses arrow-go's kernels as the reference for
its own reductions and group-bys currently has to write those references as plain Go loops.

**Component(s)**

Go, Compute

Contributor guide

Open the contributing guide

Research direction

Start with compute.GetFunctionRegistry and inspect how existing scalar and vector functions are registered, along with the FuncScalarAgg and FuncHashAgg kinds. Check the compute package documentation for the supported-function statement. Done means the documentation accurately describes aggregate support and the requested initial aggregate functions are registered and discoverable.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.