apache / apache/lucene

separate chararrayset interface from impl [LUCENE-2227]

Open
#3,303 2 comments 0 reactions 0 assignees View on GitHub
affects-version:3.0 legacy-jira-fix-version:4.9 legacy-jira-fix-version:6.0 legacy-jira-label:dead legacy-jira-priority:Minor module:analysis type:task
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

CharArraySet should be abstract
the hashing implementation currently being used should instead be called CharArrayHashSet

currently our 'CharArrayHashSet' is hardcoded across Lucene, but others might want their own impl.
For example, implementing CharArraySet as DFA with org.apache.lucene.util.automaton gives faster contains(char[], int, int) performance, as it can do a 'fast fail' and need not hash the entire string.

This is useful as it speeds up indexing in StopFilter.

I did not think this would be faster but i did benchmarks over and over with the reuters corpus, and it is, even with english text's wierd average word length of 5

---
Migrated from [LUCENE-2227](https://issues.apache.org/jira/browse/LUCENE-2227) by Robert Muir (@rmuir), updated May 09 2016

Contributor guide

Open the contributing guide

Research direction

Start by locating CharArraySet, the current CharArrayHashSet implementation, and their uses across Lucene, including StopFilter. Review the org.apache.lucene.util.automaton example described in the issue, then verify that the hashing implementation is separated and an alternate implementation can provide contains(char[], int, int) without changing the intended behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.