apache / apache/gluten

[VL] Improve sort operator performance

Open
#2,002 0 comments 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
Currently gluten will use `std::stable_sort` in Window/SMJ/Orderby cases. The `stable_sort` is not optimal and it maybe slower than vanilla Spark in some cases.

**Describe the solution you'd like**
The performance could be improved by several factors:

- Std::sort -> tim_sort
- Null check is not optimal - could skip null check if no null in row vector
- Single key/multiple keys sort
- Enable radix sort for numeric types - radix sort is much faster
- Do inplace sort if no sort payloads

**Describe alternatives you've considered**
N/A

**Additional context**
N/A

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.