matrixorigin / matrixorigin/matrixone

[Tech Request]: Optimize aggregate functions with distinct keyword

Open
#9,049 0 comments 0 reactions 2 assignees Claimed by @iamlinjunhong View on GitHub
kind/tech-request priority/p0 severity/s0 source/dev
Dominant language
Go
Stars
1.9k
Forks
311
Avg merge
1d 3h
Merged PRs (30d)
768

Description

### Is there an existing issue for the same feature request?

- [X] I have checked the existing issues.

### Is your feature request related to a problem?

```Markdown
Current implementation of distinct aggregate is using hash-table in hash-table.It's inefficient, in both memory and performance.

On my desktop machine with 32GB memory, TPCH SF=100 q16 causes OOM due to this bad implementation.
```

### Describe the feature you'd like

In plan, we should rewrite AGG nodes which have distinct aggregate function with multiple AGG nodes.

* [x] max/min/any_value (just omit DISTINCT keyword)
* [x] count/sum
* [ ] avg
* [ ] distinct aggregate coexists with normal aggregates
* [ ] multiple distinct aggregates

### Describe implementation you've considered

_No response_

### Documentation, Adoption, Use Case, Migration Strategy

_No response_

### Additional information

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.