[C++][Python] sort_by produces incorrect result
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
### Describe the bug, including details regarding any error messages, version, and platform.
To reproduce:
```python
import pyarrow
import numpy as np
print(pyarrow.__version__)
N = 2**30 // 4
data = {
"id": [4, 3, 2, 1],
"data": [np.zeros(N, dtype=np.int64) + i for i in range(4)],
}
table = pyarrow.Table.from_pydict(data)
table2 = table.sort_by("id")
print(table2)
```
Actual output:
```
pyarrow.__version__ = '19.0.1'
pyarrow.Table
id: int64
data: list
child 0, item: int64
----
id: [[1,2,3,4]]
data: [[[1,1,1,1,1,...,1,1,1,1,1],[0,0,0,0,0,...,0,0,0,0,0],[1,1,1,1,1,...,1,1,1,1,1],[0,0,0,0,0,...,0,0,0,0,0]]]
```
Changing dtype to `np.int8` gives the correct output:
```
pyarrow.__version__ = '19.0.1'
pyarrow.Table
id: int64
data: list
child 0, item: int8
----
id: [[1,2,3,4]]
data: [[[3,3,3,3,3,...,3,3,3,3,3],[2,2,2,2,2,...,2,2,2,2,2],[1,1,1,1,1,...,1,1,1,1,1],[0,0,0,0,0,...,0,0,0,0,0]]]
```
My guess is that it overflows when calculating offsets during sorting, although I have no idea how pyarrow works internally.
### Component(s)
Python
Contributor guide
Assessment
This issue has not been assessed yet.