apache / apache/lucene

Flexible "standard" query parser parses on whitespace [LUCENE-7315]

Open
#8,369 4 comments 0 reactions 1 assignee Claimed by @sarowe View on GitHub
legacy-jira-priority:Major module:queryparser type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

Copied from #3679:

The queryparser parses input on whitespace, and sends each whitespace separated term to its own independent token stream.
This breaks the following at query-time, because they can't see across whitespace boundaries:

n-gram analysis
shingles
synonyms (especially multi-word for whitespace-separated languages)
languages where a 'word' can contain whitespace (e.g. vietnamese)

Its also rather unexpected, as users think their charfilters/tokenizers/tokenfilters will do the same thing at index and querytime, but in many cases they can't. Instead, preferably the queryparser would parse around only real 'operators'.

---
Migrated from [LUCENE-7315](https://issues.apache.org/jira/browse/LUCENE-7315) by Steven Rowe (@sarowe), 2 votes, updated Jul 20 2016
Attachments: [LUCENE-7315.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-7315/LUCENE-7315.patch)
Linked issues:
- #3679

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.