apache / apache/datafusion-comet
Merge on sort keys even when they are not selected
Open
area:scan
enhancement
- Dominant language
- Scala
- Stars
- 1.3k
- Forks
- 373
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 198
Description
### What is the problem the feature request solves?
Follow up to #5323
Today, if a sort column isn't in the query's projection, we skip the merge and read the files unordered. We should read the sort columns anyway (even when not selected), merge on them, and drop them before returning the rows.
A sorted table should be able to report its ordering and merge when the query doesn't select the sort key.
### Describe the potential solution
_No response_
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.