Eliminate limit when doing scalar aggregation query
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Is your feature request related to a problem or challenge?
In scalar aggregation queries, the result is always a single row. As a result, applying a LIMIT operator above the Aggregate operator has no effect unless the limit is set to 0. Therefore, I propose we can safely eliminate unnecessary LIMIT operators in scalar aggregation queries.
For example, given a sql
``` sql
select count(*) from t limit 10
```
The `count` UDF always returns a single row and applying a `limit 10` is definitely unnecessary. However, the execution plan has a top `GlobalLimitExec` operator.
```
+---------------+------------------------------------------------------------------------------+
| plan_type | plan |
+---------------+------------------------------------------------------------------------------+
| logical_plan | Limit: skip=0, fetch=10 |
| | Aggregate: groupBy=[[]], aggr=[[count(Int64(1)) AS count(*)]] |
| | TableScan: information_schema.tables projection=[] |
| physical_plan | GlobalLimitExec: skip=0, fetch=10 |
| | AggregateExec: mode=Final, gby=[], aggr=[count(*)] |
| | CoalescePartitionsExec |
| | AggregateExec: mode=Partial, gby=[], aggr=[count(*)] |
| | RepartitionExec: partitioning=RoundRobinBatch(8), input_partitions=1 |
| | StreamingTableExec: partition_sizes=1 |
| | |
+---------------+------------------------------------------------------------------------------+
```
### Describe the solution you'd like
I think we can design an optimizer rule (logical or physical ??) , to eliminate such unnecessary LIMIT operators.
### Describe alternatives you've considered
_No response_
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.