matrixorigin / matrixorigin/matrixone
[Enhancement]: Simplify constant ORDER BY and HAVING after single-row aggregate elimination
- Dominant language
- Go
- Stars
- 1.9k
- Forks
- 311
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 768
Description
## Summary
After #27850 proves that a complete-key group contains one row and rewrites aggregates to row expressions, run a proof-based simplification pass over ORDER BY and HAVING. This can unlock the scan reduction that a remaining Sort or always-true Filter currently blocks.
## High-value example
```sql
select id, count(*) c
from t
group by id
order by c desc
limit 10;
```
With complete primary/unique-key grouping, `COUNT(*)` is constant `1` for every output row. The current #27850 plan removes Aggregate but retains Sort, so it still scans and sorts the full relation. If every order key is proven constant, all rows tie and SQL without an additional tie-breaker may return any ten rows; Sort is redundant and LIMIT can reach the scan.
This is expected to move eligible ordered cases from hash-only improvement toward the same O(K + OFFSET) scan behavior as unordered LIMIT.
## Candidate simplifications
- remove Sort when every ORDER BY expression folds to a query-constant value after aggregate remapping;
- remove an always-true HAVING predicate such as `COUNT(*) = 1`;
- replace an always-false HAVING predicate with an empty result without scanning, when error/volatile evaluation cannot be suppressed;
- rewrite `ORDER BY MIN(pk)`/`MAX(pk)` to the proven row/key expression, allowing an existing ordered access path to consume it;
- propagate constants through aliases, deterministic casts and tuples without duplicating volatile expressions.
## Required guards
Fail closed for:
- any non-constant remaining sort key or explicit deterministic tie-breaker;
- `WITH TIES`, rank/window semantics or another consumer that observes the complete tied stream;
- `SQL_CALC_FOUND_ROWS` when early scan termination would change `FOUND_ROWS()`;
- volatile/side-effecting expressions or expressions whose evaluation errors would be suppressed;
- NULL/collation/coercion behavior not preserved by folding;
- ORDER BY aggregate results whose single-row expression varies per row, such as `SUM(v)`;
- any query where uniqueness or single-row aggregate conversion was not already proven.
The rule must inspect the rewritten expression/property, not special-case `COUNT(*)` text.
## Validation matrix
- one/multiple ORDER BY keys: all constant, mixed constant/non-constant, ASC/DESC, NULLs and aliases;
- LIMIT 0/1/K, large K, OFFSET, prepared LIMIT and `SQL_CALC_FOUND_ROWS`;
- `COUNT(*)`, nullable COUNT, MIN/MAX of key, SUM/AVG of varying columns;
- always-true/false HAVING, conjunction/disjunction and volatile controls;
- primary key, non-PK unique key and ineligible no-PK controls;
- one/multi-CN and ordered/unordered scan access paths;
- exact result multisets for unordered ties and exact results when a tie-breaker exists.
## Performance acceptance
On the same NVMe data with only the binary changed:
1. demonstrate that `GROUP BY unique-key ORDER BY COUNT(*) LIMIT K` no longer scans/sorts N rows;
2. report scan rows/bytes, Sort input/retained rows, CPU, peak memory and wall-time medians;
3. include varying aggregate-order controls that retain Sort and full scan;
4. require a large improvement for the all-constant case and no material regression for controls.
## Related
- Primary-key aggregate elimination: #27850
- Non-PK unique-key proof: #27856
- Storage/index order property: #27857
- Umbrella decision: #27730
Contributor guide
Assessment
This issue has not been assessed yet.