IndexableField changes its IndexableFieldType when the index is re-opened for reading [LUCENE-7171]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
This code:
```Java
/* Store one document into an index */
Directory index = new RAMDirectory();
IndexWriterConfig config = new IndexWriterConfig(new StandardAnalyzer());
IndexWriter w = new IndexWriter(index, config);
Document d1 = new Document();
d1.add(new StringField("isbn", "9900333X", Field.Store.YES));
w.addDocument(d1);
w.commit();
w.close();
/* inspect IndexableFieldType */
IndexableField f1 = d1.getField("isbn");
System.err.println("FieldType for " + f1.stringValue() + " : " + f1.fieldType());
/* retrieve all documents and inspect IndexableFieldType */
IndexSearcher s = new IndexSearcher(DirectoryReader.open(index));
TopDocs td = s.search(new MatchAllDocsQuery(), 1);
for (ScoreDoc sd : td.scoreDocs) {
Document d2 = s.doc(sd.doc);
IndexableField f2 = d2.getField("isbn");
System.err.println("FieldType for " + f2.stringValue() + " : " + f2.fieldType());
}
```
Produces:
```Java
FieldType for 9900333X : stored,indexed,omitNorms,indexOptions=DOCS
FieldType for 9900333X : stored,indexed,tokenized,omitNorms,indexOptions=DOCS
```
The `StringField` field `isbn` is not tokenized, as correctly reported by the first output, which happens right after closing the writer.
However, it becomes tokenized when the index is re-opened with a new reader.
---
Migrated from [LUCENE-7171](https://issues.apache.org/jira/browse/LUCENE-7171) by Roberto Cornacchia, updated May 23 2016
Contributor guide
Research direction
Start by running the Java reproduction in the issue and compare the IndexableFieldType before and after opening the index with a reader. Trace how the StringField is reconstructed during document retrieval; done means the field type remains consistent and the reproduction reports the expected non-tokenized type.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100