Numeric DV block compression ignores the gcd when computing the number of bits required [LUCENE-9899]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
When numeric doc values are splitted per block we compute the number of bits per value [from the minimum and maximum value present in the block](https://github.com/apache/lucene/blob/d5d6dc079395c47cd6d12dcce3bcfdd2c7d9dc63/lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesConsumer.java#L390). However, the greatest common divisor is not taken into account so the number is overvalued for cases where it is greater than 1.
---
Migrated from [LUCENE-9899](https://issues.apache.org/jira/browse/LUCENE-9899) by Jim Ferenczi (@jimczi), updated Apr 01 2021
Attachments: [LUCENE-9899.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-9899/LUCENE-9899.patch)
Contributor guide
Research direction
Start in lucene/core/src/main/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesConsumer.java at the linked computation around line 390. Trace how the block's minimum and maximum values determine the required bits, then inspect the issue's attached LUCENE-9899.patch for context. Done means numeric doc-value blocks account for their greatest common divisor when calculating the bit width.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100