matrixorigin / matrixorigin/matrixone
[Performance]: Complete special-operator pruning and use low-cost row carriers
- Dominant language
- Go
- Stars
- 1.9k
- Forks
- 311
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 768
Description
## Is there an existing issue for performance?
- [x] I have checked the existing issues.
## Environment
- Version or commit-id: main at `7821d21c72`
- Hardware parameters: N/A
- OS type: Linux
- Others: logical-plan column pruning in `QueryBuilder.remapAllColRefs`
## Details of Performance
After closing the general `UNION ALL`, `AGG`, and unused-window gaps, several special operators and row-carrier fallbacks still retain avoidable work:
1. `SAMPLE` marks every sample expression as required even if only a subset of sampled outputs is consumed.
2. `TIME_WINDOW` retains every aggregate expression and input even when an outer query discards some aggregate outputs.
3. Zero-column fallbacks choose position/column/aggregate 0 for table/function scans, `UNION ALL`, and scalar aggregates. The first candidate can be a wide varlen column or an expensive aggregate such as `group_concat`, causing avoidable I/O, allocation, and CPU work.
The target is to minimize physical input width and operator state without changing row-count semantics. In particular, `SAMPLE` and scalar aggregation may still need a row-producing carrier when all values are discarded.
## Additional information
Proposed design constraints:
- Model row cardinality as a first-class requirement rather than implicitly retaining slot 0.
- Prefer zero-column batches if all affected execution operators support them; otherwise choose a deterministic low-cost carrier using type width and aggregate cost.
- Compact `SAMPLE`/`TIME_WINDOW` physical slots and remap `HAVING`, special `_wstart`/`_wend` outputs, merge-stage state, and parent projections together.
- Preserve side-effecting expressions and all group/order/partition dependencies.
- Add plan-shape tests, empty-input and 0/1/many-row result tests, wide-varlen performance harnesses, and allocation/peak-memory measurements.
Related architectural work: #14077.
Contributor guide
Assessment
This issue has not been assessed yet.