Enable extraction of originating term for ICU collation keys [LUCENE-4799]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
By concatenating generated ICU collation keys bytes with the originating term, it is possible to extract the originating term at a later time. This makes it possible to build a collator sorted facet field and similar multi-value/document structures.
ICU collation keys are guaranteed to be terminated by a 0 (https://ssl.icu-project.org/apiref/icu4j48rc1/com/ibm/icu/text/CollationKey.html) and since comparison of keys stop when a 0 is encountered, the addition of the originating term does not affect sort order. As 0 are _only_ used for termination in the key bytes, the extraction of the originating term is unambiguous.
---
Migrated from [LUCENE-4799](https://issues.apache.org/jira/browse/LUCENE-4799) by Toke Eskildsen (@tokee), 1 vote
Attachments: [LUCENE-4799.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-4799/LUCENE-4799.patch)
Contributor guide
Research direction
Start by reviewing the ICU CollationKey behavior described in the issue and the attached LUCENE-4799.patch. Trace how Lucene generates and compares ICU collation key bytes, then determine where the originating term should be appended and extracted. Done means the term can be recovered unambiguously without changing collation sort order.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100