apache / apache/lucene

Flexible configuration for DocValue compressions [LUCENE-9737]

Open
#10,776 0 comments 0 reactions 0 assignees View on GitHub
affects-version:9.0 legacy-jira-priority:Major module:core/codecs type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

#10251 added compression for Binary DocValues.

#10702 added compression for Terms Dict from SortedSet/Sorted DocValues.

#10418 made Binary DocValues compression configurable, it introduced two modes for DocValues(Similar with the two modes from Lucene87StoredFieldsFormat): 
- BEST_SPEED
- BEST_COMPRESSION

It's simple to use...so Terms-dict compression shares the same configuration.

But some users may only want to try Terms Dict compression, since Binary DocValue compression may cause performance reduction..

So we need to make configuration flexible.

---
Migrated from [LUCENE-9737](https://issues.apache.org/jira/browse/LUCENE-9737) by Jaison.Bi, updated Mar 01 2021
Linked issues:
- #10702

Pull requests: https://github.com/apache/lucene-solr/pull/2412

Contributor guide

Open the contributing guide

Research direction

Review the compression changes referenced by #10251, #10702, and #10418, along with pull request #2412, to understand how BEST_SPEED and BEST_COMPRESSION are currently shared. Done means the issue's requested configuration flexibility is supported without forcing Binary DocValues and Terms Dict compression to use the same setting.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, search
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.