Should Packed64 use a byte[] plus VarHandles? [LUCENE-10205]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
By being backed by a long[], Packed64 often has to merge bits coming from two different longs. If it was backed by a byte[], it could always read a single long, which would help remove conditionals?
The main downside is that we'd need paging to support high value counts with high numbers of bits (when value_count \* bits_per_value / 8 > ArrayUtil.MAX_ARRAY_LENGTH).
---
Migrated from [LUCENE-10205](https://issues.apache.org/jira/browse/LUCENE-10205) by Adrien Grand (@jpountz), updated Jan 20 2022
Contributor guide
Research direction
Start by locating Packed64 and ArrayUtil.MAX_ARRAY_LENGTH, then read how Packed64 currently stores and merges values. Investigate whether a byte[] with VarHandles can avoid cross-long conditionals and how paging would address the stated array-size limit. Done means the representation change and its high-value-count implications have a resolved design.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100