apache / apache/datafusion

Optimize `sort_batch` for single column case

Open
#14,475 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

`sort_batch` always uses multi column sort by sorting first to indices and then sorting the batch. See:

https://github.com/apache/datafusion/blob/ea788c72dddf24acd8dc8068cede939e11dfab84/datafusion/physical-plan/src/sorts/sort.rs#L628

We can use https://arrow.apache.org/rust/arrow/compute/fn.sort_limit.html instead

### Describe the solution you'd like

For single row case, sort the batch directly instead of sorting multiple columns.

### Describe alternatives you've considered

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Read datafusion/physical-plan/src/sorts/sort.rs around line 628 and the Arrow Rust sort_limit documentation. Confirm whether the request targets a single-column sort despite the body saying “single row,” then compare the direct path with the existing multi-column path and verify that sorting behavior is preserved.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
performance
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.