Optimize infallible dictionary key casting in `try_new_from_builder` method
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 1.3k
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 167
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
**Describe the solution you'd like**
In https://github.com/apache/arrow-rs/pull/7611 we added a new method to the dictionary builders called `try_new_from_builder` that allows creating a new builder with the same values, but a different type for the dictionary keys.
This method uses the `PrimitiveArray::try_unary` method to do the casting, but when casting infallible types (e.g. going from u8 -> u16 keys), it would probably be faster to use `PrimitiveArray::unary_mut`.
We should improve the implementation of `try_new_from_builder` to use `unary_mut` where possible. It might also be good to have add benchmark for this method as part of this work, so we could measure if there really is a performance improvement.
**Describe alternatives you've considered**
**Additional context**
Discussion https://github.com/apache/arrow-rs/pull/7611/files#r2143611512
Contributor guide
Research direction
Start by reading the `try_new_from_builder` implementation and the `PrimitiveArray::try_unary` and `unary_mut` entry points, then review PR 7611 for the builder context. Update the casting path for infallible key conversions and add a benchmark if the existing benchmark structure supports it; done means the implementation is covered by tests and the performance can be compared.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- performance
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100