apache / apache/lucene

span query matches too many docs when two query terms are the same unless inOrder=true [LUCENE-3120]

Open
#4,193 10 comments 0 reactions 0 assignees View on GitHub
legacy-jira-fix-version:4.9 legacy-jira-fix-version:6.0 legacy-jira-priority:Minor module:core/search type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

spinoff of user list discussion - [SpanNearQuery - inOrder parameter](http://markmail.org/message/i4cstlwgjmlcfwlc).

With 3 documents:
- "a b x c d"
- "a b b d"
- "a b x b y d"

Here are a few queries (the number in parenthesis indicates expected #hits):

These ones work **as expected**:
- (1) in-order, slop=0, "b", "x", "b"
- (1) in-order, slop=0, "b", "b"
- (2) in-order, slop=1, "b", "b"

These ones match **too many** hits:
- (1) any-order, slop=0, "b", "x", "b"
- (1) any-order, slop=1, "b", "x", "b"
- (1) any-order, slop=2, "b", "x", "b"
- (1) any-order, slop=3, "b", "x", "b"

These ones match **too many** hits as well:
- (1) any-order, slop=0, "b", "b"
- (2) any-order, slop=1, "b", "b"

Each of the above passes when using a phrase query (applying the slop, no in-order indication in phrase query).

This seems related to a known overlapping spans issue - [non-overlapping Span queries](http://markmail.org/message/7jxn5eysjagjwlon) - as indicated by Hoss, so we might decide to close this bug after all, but I would like to at least have the junit that exposes the behavior in JIRA.

---
Migrated from [LUCENE-3120](https://issues.apache.org/jira/browse/LUCENE-3120) by Doron Cohen, updated May 09 2016
Attachments: [LUCENE-3120.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-3120/LUCENE-3120.patch) (versions: 3)
Linked issues:
- #6994

Contributor guide

Open the contributing guide

Research direction

Start with the SpanNearQuery behavior described in the issue and compare the any-order cases with the phrase-query cases. Review the attached LUCENE-3120.patch and the linked discussion about non-overlapping Span queries, then add the requested JUnit coverage for the listed documents and expected hit counts. Done means the tests expose the duplicate-term matching behavior and its disposition is clear.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.