Specialized `GroupValues` for `primitive` and `large_primitive`
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Is your feature request related to a problem or challenge?
After #15961, in `GroupValuesPrimitive`, we store the `hash` of group to speed up rehashing.
However, I found the bottleneck of `rehashing` is not hash computation, it is actually randomly access to the `values`.
So, after experimenting, the better is:
- We directly store the `group value` in `map`
- So, during probing we can perform `eq` in-place
- And, during rehashing we just perform the cheap hash computation for `group value`
But for the `large primitives`, it may be still better to store their hashes in `map` rather than themselves, otherwise it will make `map` too large and lead to possible regression.
### Describe the solution you'd like
_No response_
### Describe alternatives you've considered
_No response_
### Additional context
_No response_
Contributor guide
Research direction
Start by tracing GroupValuesPrimitive and its map access during probing and rehashing. Compare storing group values with storing hashes for large primitives, then validate that the specialized primitive and large_primitive behavior improves rehashing without making the map too large or causing a regression.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100