Should we not enlarge PagedGrowableWriter initial bitPerValue on NodeHash.rehash()?
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
### Description
Spawn from https://github.com/apache/lucene/pull/12738
It seems on [rehash](https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/util/fst/NodeHash.java#L310C1-L310C56) we will use an initial bitPerValue set to cover the last node address.
Does that mean every values, including the ones with low-address will use the same bpv as the high-address nodes? PagedGrowableWriter already enlarge the bpv [automatically](https://github.com/apache/lucene/blob/8fa0de2743e87dd264619632978c8dc68323947a/lucene/core/src/java/org/apache/lucene/util/packed/GrowableWriter.java#L86), so maybe we could always set the initial bpv to a small value (like 8?)
Let see if the total size is different when making the change
### Version and environment details
_No response_
Contributor guide
Research direction
Start in lucene/core/src/java/org/apache/lucene/util/fst/NodeHash.java at rehash(), then read PagedGrowableWriter and the linked GrowableWriter implementation to understand initial bitPerValue growth. Compare total size with a smaller initial value such as 8 and determine whether the change preserves behavior while reducing memory use.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100