GraphTokenStreamFiniteStrings does not handle position inc > 1 in multi-word synoyms [LUCENE-8137]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
The automaton built for graph queries that contain multiple multi-word synonyms does not handle gaps if they appear in the middle of a multi-word synonym. In such case the token next to the gap is considered as part of the multi-word synonym.
Stop words that appear before or after multi-word synonyms are handled correctly in the current version but the synonym rule "part of speech, pos" for instance does not create the expected query if "of" is removed by a filter that is set after the synonym_graph. One solution would be to reuse TokenStreamToAutomaton (with minor changes to add the ability to create token transitions rather than chars) which preserves gaps (as a transition) in the produced automaton.
---
Migrated from [LUCENE-8137](https://issues.apache.org/jira/browse/LUCENE-8137) by Jim Ferenczi (@jimczi), 3 votes, updated Oct 10 2019
Attachments: [SGF_SF_interaction.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-8137/SGF_SF_interaction.patch)
Linked issues:
- [SOLR-11968](https://issues.apache.org/jira/browse/SOLR-11968)
Contributor guide
Assessment
This issue has not been assessed yet.