apache / apache/arrow-rs

Unstable (lex)sort_to_indices

Open
#4,545 17 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Rust
Stars
3.6k
Forks
1.3k
Avg merge
2d 18h
Merged PRs (30d)
169

Description

**Describe the bug**

Arrow ordering uses the unstable sort which doesn't guarantee sort order between same elements, this causing undesired behavior for systems that expects stable sort

**To Reproduce**

```
#[test]
fn test_sort_primitives2() {
let n = 20;

let mut arr: [(u32, i32); 20] = [(0, 0); 20];

(0 .. n).map(|d| Some(d % 2)).enumerate().for_each(|v| arr[v.0] = (v.0.try_into().unwrap(), v.1.unwrap()));
//println!("{:?}", arr);
sort_unstable_by::<(u32, i32), _>(&mut arr, n as usize, |a, b| cmp(a.1, b.1).reverse());
println!("{:?}", arr.iter().filter(|v| v.1 == 1).map(|v| v.0).collect::>());
}
```
The test outputs array indices for 1 and this is correct, in sync with input data
```
[1, 3, 5, 7, 9, 11, 13, 15, 17, 19]
```

For n = 21 which shouldn't change the output, but instead indices are messed
```
[15, 1, 3, 5, 7, 9, 11, 13, 17, 19]
```
**Expected behavior**

**Additional context**

Related to https://github.com/apache/arrow-rs/issues/2871

Contributor guide

Open the contributing guide

Research direction

Start at the lexsort_to_indices entry point and inspect how it uses sort_unstable_by; compare the behavior with the reproduction shown in the issue and related issue #2871. Add or run a regression test covering duplicate keys for both input sizes, and consider the work done when equal elements retain their original order.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.