apache / apache/datafusion-comet

Add support for SortAggregateExec

Open
#1,994 2 comments 0 reactions 0 assignees View on GitHub
enhancement performance
Dominant language
Scala
Stars
1.3k
Forks
373
Avg merge
2d 4h
Merged PRs (30d)
198

Description

### What is the problem the feature request solves?

I was experimenting with some benchmarking and noticed that we do not yet have a native version of `SortAggregateExec`. This resulted in poor performance due to `CometColumnarToRow` being inserted into the plan.

```
Execute InsertIntoHadoopFsRelationCommand [COMET: Execute InsertIntoHadoopFsRelationCommand is not supported]
+- WriteFiles [COMET: WriteFiles is not supported]
+- Window [COMET: Partitioning and sorting specifications must be the same.]
+- CometSort
+- AQEShuffleRead
+- CometSinkPlaceHolder
+- CometColumnarExchange
+- SortAggregate [COMET: SortAggregate is not supported]
+- CometColumnarToRow
+- CometSort
+- AQEShuffleRead
+- CometColumnarExchange
+- SortAggregate [COMET: SortAggregate is not supported]
+- CometColumnarToRow
+- CometSort
+- CometScan parquet
```

### Describe the potential solution

We should see what is involved in implementing `SortAggregateExec` natively.

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.