[Feature] Compare batch maxKey to reduce comparisons in SortMergeReader
Open
enhancement
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/incubator-paimon/issues) and found nothing similar.
### Motivation
In SortMergeReader, each sub reader is batched reader.
When adding a new batch to the priority queue, we can look at the maximum key of the batch, and if its maximum key is smaller than the minimum key of other batches, then we can just output the whole batch.
### Solution
_No response_
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.