Refactoring multiterm analysis [LUCENE-5470]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
There are currently three methods to analyze multiterms in Lucene and Solr:
1) QueryParserBase
2) AnalyzingQueryParser
3) TextField (Solr)
The code in QueryParserBase and in TextField do not consume the tokenstream if more than one token is generated by the analyzer. (Admittedly, thanks to the magic of MultitermAwareComponents in Solr, this type of exception probably never happens and the unconsumed stream problem is probably non-existent in Solr.)
I propose consolidating the multiterm analysis code into one place: QueryBuilder in Lucene core.
This is part of a refactoring that will also help reduce duplication of code with #6269.
---
Migrated from [LUCENE-5470](https://issues.apache.org/jira/browse/LUCENE-5470) by Tim Allison (@tballison), 1 vote, updated Apr 09 2014
Attachments: [LUCENE-5470_QBuilder.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-5470/LUCENE-5470_QBuilder.patch), [LUCENE-5470_QPBase.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-5470/LUCENE-5470_QPBase.patch), [LUCENE-5470.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-5470/LUCENE-5470.patch)
Contributor guide
Research direction
Compare the multiterm analysis implementations in QueryParserBase, AnalyzingQueryParser, and Solr's TextField, then read the attached LUCENE-5470 patches and the QueryBuilder entry point. The intended result is a single QueryBuilder location for this analysis, including handling streams that produce multiple tokens and reducing duplication with issue #6269.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100