apache / apache/arrow

[C++] binary_slice with its default `stop` raises "Negative buffer resize"

Open
#51,222 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

**Describe the bug**

`pyarrow.compute.binary_slice(arr, start)` with `stop` left at its default raises
`ArrowInvalid: Negative buffer resize` once a value is longer than a few hundred bytes. Passing an
explicit large `stop` works, and `utf8_slice_codeunits` handles the same default correctly since
#36575; the binary kernel still has the overflow in its output-size arithmetic.

pyarrow 25.0.1, Python 3.13.9, macOS 26.6 (arm64), wheel from PyPI.

```python
import pyarrow as pa, pyarrow.compute as pc

b = pa.array([b"abcdef", b"x" * 300], pa.binary())
print([len(x) for x in pc.binary_slice(b, 1, 1000).to_pylist()]) # explicit stop: fine
print([len(x) for x in pc.utf8_slice_codeunits(pa.array(["abcdef", "x" * 300]), 1).to_pylist()])
print(pc.binary_slice(b, 1).to_pylist()) # default stop
```

Output:

```
[5, 299]
[5, 299]
Traceback (most recent call last):
...
pyarrow.lib.ArrowInvalid: Negative buffer resize: -4
```

**Expected behavior**

`binary_slice(b, 1)` returns `[b"bcdef", b"x" * 299]`, as `utf8_slice_codeunits` does for the string
equivalent.

Related: #34929 (umbrella for slice-option overflow), #36575 (the utf8 fix).

**Component(s)**

C++, Python

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the Python example with pyarrow.compute.binary_slice, then inspect the binary_slice kernel and the related utf8_slice_codeunits fix in #36575. Compare the default-stop output-size arithmetic and add coverage for values longer than a few hundred bytes; done means binary_slice(arr, start) returns the expected sliced byte strings without the negative buffer resize error.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
data-engineering
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
75/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.