Add API to avoid allocations when using `ArrayData::slice()` on owned ArrayData
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 1.3k
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 167
Description
> we can probably avoid an additional allocation for sliced arrays by making a version of `slice()` that consumes self -- like `sliced()` perhaps 🤔
_Originally posted by @alamb in [#9114](https://github.com/apache/arrow-rs/pull/9114/changes#r2670379840)_
The idea is that calling `ArrayData::slice()` creates a *new* array data (needs to allocate a Vec for the new child buffers). For some cases, the caller owns the ArrayData and thus could simply reuse the allocation by taking it as owned
Something like
```rust
impl ArrayData {
/// Like Self::sliced, but takes an owned self, and reuses the allocations, making it more efficient
fn sliced(self, offset: usize, len: usize) -> Self {
...
}
}
```
Contributor guide
Research direction
Start at the ArrayData::slice() implementation and trace how slicing constructs child buffers and allocates its Vec. Define the owned-self slicing API described in the issue, preserving the existing slice behavior while reusing allocations where possible. Done means callers can slice an owned ArrayData without the additional allocation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100