getOffsetGap should not be called for non-anaylyzed fields [LUCENE-2801]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
from: #3311
Since Lucene 3.0.3, when a PerFieldAnalyzerWrapper is constructed with a null defaultAnalyzer it will NPE when DocInverterPerField calls:
```Java
fieldState.offset += docState.analyzer.getOffsetGap(field);
```
This block should first check that the field is analyzed, or the javadoc on PerFieldAnalyzerWrapper could mention that a null defaultAnalyzer is disallowed.
Also, the main reason for checking for isAnalyzed, from Uwe Schindler in #3311
> One problem coming from not checking for "analyzed" is this:
> You add a field indexed and it gets analyzed by PFAW - After that you add the same field name stored-only (which is perfectly legal and often used, e\.g\. when the stored value is binary or in some other format and does not correspond to the indexed text), the positionIncrement is increased\. After that you again add another instance of the same field as indexed-only, which also increases posIncr\. So you have 2 times the gap between both indexed sub-fields\. This is definitely wrong\.
---
Migrated from [LUCENE-2801](https://issues.apache.org/jira/browse/LUCENE-2801) by Nick Pellow
Linked issues:
- #3311
Contributor guide
Research direction
Start in DocInverterPerField at the getOffsetGap call and read how PerFieldAnalyzerWrapper handles a null defaultAnalyzer and non-analyzed fields. Compare the behavior described in linked issue #3311; done means avoiding the null-analyzer failure and preventing offset or position gaps for non-analyzed field instances.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100