TokenStream contract violation: reset()/close() call missing, [LUCENE-7227]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
public static void highLightClojure(int id, String text, String field) {
try {
Query queryToSearch;
TokenStream tokenStream = TokenSources.getTokenStream( field,text, analyzer);
queryToSearch = new QueryParser("asddf", analyzer).parse("read text file string utf8");
Highlighter highlighter = new Highlighter(new SimpleHTMLFormatter(),
new QueryScorer(queryToSearch));
TextFragment[] frag = highlighter.getBestTextFragments(tokenStream, text, false, 4);
for (int j = 0; j < frag.length; j++) {
if ((frag[j] != null)) {
System.out.println("score: " + frag[j].getScore() + ", frag: " + (frag[j].toString()));
}
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (InvalidTokenOffsetsException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (ParseException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
Exception:
TokenStream contract violation: reset()/close() call missing,
reset() called multiple times, or subclass does not call
super.reset(). Please see Javadocs of TokenStream class for more
information about the correct consuming workflow.
The tokenStream must be created after QueryParser, its an implicit dependency, but no any information mentioned it.
---
Migrated from [LUCENE-7227](https://issues.apache.org/jira/browse/LUCENE-7227) by Lei Zhang
Contributor guide
Research direction
Start with the highLightClojure entry point shown in the report and reproduce the TokenStream contract exception. Trace the TokenSources, QueryParser, and Highlighter interaction to determine the expected consuming workflow; done means the reported highlighting path completes without the reset()/close() violation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100