apache / apache/datafusion

Implement Aggregate function `map_agg`

Open
#22,993 1 comment 0 reactions 1 assignee Claimed by @LiaCastaneda View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

DataFusion currently has no support for aggregate function `map_agg` , which is supported in other engines like [trino](https://trino.io/docs/current/functions/aggregate.html#map_agg) , [presto](https://prestodb.io/docs/current/functions/aggregate.html#map_agg), spark [map_from_arrays](https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.functions.map_from_arrays.html) (not UDAF), [json_object_agg](https://www.postgresql.org/docs/current/functions-aggregate.html) in postgres

this is also the equivalent of `array_agg` but for `map` type

### Describe the solution you'd like

Extend the api `AggregateUDFImpl` to implement `MapAgg`.

The expected behavior would be:

```
SELECT k, map_agg(v, m)
FROM (VALUES
('a', 1, 10),
('a', 2, 20),
('b', 3, 30)
) t(k, v, m)
GROUP BY k;
```

```
k | map_agg(v, m)
--+------------------
a | {1: 10, 2: 20}
b | {3: 30}
```

Each group collects its own (v, m) pairs into one map. Group a has two rows so its map has two entries, group b has one.

optionally, we can also support calling `ORDER BY col ` on the function. (like `postgres` and `array_agg` does)

### Describe alternatives you've considered

Implement it in our codebase, but given the generic behavior of this function and the fact that it exists in other engines, I think it would be worth upstreaming.

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.