Use run-length encoding when writing docIds in BKD tree [LUCENE-9178]
Open
legacy-jira-priority:Major
type:enhancement
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
I think we can easily check if it make sense to write docIds using length compression in the BKD tree. This can probably save some space in the case of Muti value documents, e.g LatLonShape and XYShape.
---
Migrated from [LUCENE-9178](https://issues.apache.org/jira/browse/LUCENE-9178) by Ignacio Vera (@iverase)
Contributor guide
Research direction
Start by locating the BKD tree code that writes docIds, then inspect how it is used for multi-value documents such as LatLonShape and XYShape. Determine whether run-length encoding is beneficial for those cases; done means the approach is implemented with evidence of space savings and no regressions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100