TERM1 OR NOT TERM2 does not perform as expected [LUCENE-666]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
test:
[junit] Testsuite: org.apache.lucene.search.TestAornotB
[junit] Tests run: 3, Failures: 1, Errors: 0, Time elapsed: 0.39 sec
[junit] ------------- Standard Output ---------------
[junit] Doc1 = A B C
[junit] Doc2 = A B C D
[junit] Doc3 = A C D
[junit] Doc4 = B C D
[junit] Doc5 = C D
[junit] -------------------------------------------------
[junit] With query "A OR NOT B" we expect to hit
[junit] all documents EXCEPT Doc4, instead we only match on Doc3.
[junit] While LUCENE currently explicitly does not support queries of
[junit] the type "find docs that do not contain TERM" - this explains
[junit] not finding Doc5, but does not justify elimnating Doc1 and Doc2
[junit] -------------------------------------------------
[junit] the fix shoould likely require a modification to QueryParser.jj
[junit] around the method:
[junit] protected void addClause(Vector clauses, int conj, int mods, Query q)
[junit] Query:c:a -c:b hits.length=1
[junit] Query Found:Doc[0]= A C D
[junit] 0.0 = (NON-MATCH) Failure to meet condition(s) of required/prohibited clause(s)
[junit] 0.6115718 = (MATCH) fieldWeight(c:a in 1), product of:
[junit] 1.0 = tf(termFreq(c:a)=1)
[junit] 1.2231436 = idf(docFreq=3)
[junit] 0.5 = fieldNorm(field=c, doc=1)
[junit] 0.0 = match on prohibited clause (c:b)
[junit] 0.6115718 = (MATCH) fieldWeight(c:b in 1), product of:
[junit] 1.0 = tf(termFreq(c:b)=1)
[junit] 1.2231436 = idf(docFreq=3)
[junit] 0.5 = fieldNorm(field=c, doc=1)
[junit] 0.6115718 = (MATCH) sum of:
[junit] 0.6115718 = (MATCH) fieldWeight(c:a in 2), product of:
[junit] 1.0 = tf(termFreq(c:a)=1)
[junit] 1.2231436 = idf(docFreq=3)
[junit] 0.5 = fieldNorm(field=c, doc=2)
[junit] 0.0 = (NON-MATCH) Failure to meet condition(s) of required/prohibited clause(s)
[junit] 0.0 = match on prohibited clause (c:b)
[junit] 0.6115718 = (MATCH) fieldWeight(c:b in 3), product of:
[junit] 1.0 = tf(termFreq(c:b)=1)
[junit] 1.2231436 = idf(docFreq=3)
[junit] 0.5 = fieldNorm(field=c, doc=3)
[junit] Query:c:a (-c:b) hits.length=3
[junit] Query Found:Doc[0]= A B C
[junit] Query Found:Doc[1]= A B C D
[junit] Query Found:Doc[2]= A C D
[junit] 0.3057859 = (MATCH) product of:
[junit] 0.6115718 = (MATCH) sum of:
[junit] 0.6115718 = (MATCH) fieldWeight(c:a in 1), product of:
[junit] 1.0 = tf(termFreq(c:a)=1)
[junit] 1.2231436 = idf(docFreq=3)
[junit] 0.5 = fieldNorm(field=c, doc=1)
[junit] 0.5 = coord(1/2)
[junit] 0.3057859 = (MATCH) product of:
[junit] 0.6115718 = (MATCH) sum of:
[junit] 0.6115718 = (MATCH) fieldWeight(c:a in 2), product of:
[junit] 1.0 = tf(termFreq(c:a)=1)
[junit] 1.2231436 = idf(docFreq=3)
[junit] 0.5 = fieldNorm(field=c, doc=2)
[junit] 0.5 = coord(1/2)
[junit] 0.0 = (NON-MATCH) product of:
[junit] 0.0 = (NON-MATCH) sum of:
[junit] 0.0 = coord(0/2)
[junit] ------------- ---------------- ---------------
[junit] Testcase: testFAIL(org.apache.lucene.search.TestAornotB): FAILED
[junit] resultDocs =A C D expected:<3> but was:<1>
[junit] junit.framework.AssertionFailedError: resultDocs =A C D expected:<3> but was:<1>
[junit] at org.apache.lucene.search.TestAornotB.testFAIL(TestAornotB.java:137)
[junit] Test org.apache.lucene.search.TestAornotB FAILED
---
Migrated from [LUCENE-666](https://issues.apache.org/jira/browse/LUCENE-666) by Dejan Nenov, 3 votes, updated Jan 25 2011
Environment:
```
Windows XP, JavaCC 4.0, JDK 1.5
```
Attachments: [ASF.LICENSE.NOT.GRANTED--TestAornotB.java](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-666/ASF.LICENSE.NOT.GRANTED--TestAornotB.java)
Linked issues:
- #1103
- #1150
Contributor guide
Research direction
Start with org.apache.lucene.search.TestAornotB.java, especially the failing test at line 137, and run it to reproduce the reported result. Then inspect QueryParser.jj around addClause(Vector clauses, int conj, int mods, Query q). Done means the A OR NOT B query returns the three expected documents while preserving the documented limitation for documents matching only NOT B.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100