[MSE] Support sender-side sorting for sort exchanges
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 2d 55m
- Merged PRs (30d)
- 182
Description
## Problem
`PinotLogicalSortExchange` currently sorts only on the receiver. Operators such as WINDOW and MATCH_RECOGNIZE therefore require the receiver to buffer and sort the complete exchanged input instead of merging already sorted sender streams.
## Proposed scope
- Add sender-side sorting using the exchange collation, including direction and null ordering.
- Merge sorted mailbox streams incrementally on the receiver with bounded memory.
- Preserve cancellation, deadline, backpressure, and error propagation behavior.
- Add distributed tests covering multiple senders, duplicate keys, nulls, and empty streams.
## Context
Follow-up to #19311 and the existing receiver-side-only note in `PinotWindowExchangeNodeInsertRule`.
Contributor guide
Research direction
Start by reading PinotLogicalSortExchange and the receiver-side-only note in PinotWindowExchangeNodeInsertRule, then review follow-up issue #19311. Trace the existing exchange and mailbox behavior before defining the sender sort and receiver merge boundaries. Done means distributed tests cover multiple senders, duplicate keys, nulls, and empty streams while cancellation, deadlines, backpressure, errors, and bounded memory remain correct.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100