DuplicateFilter incorrectly handles multiple calls to getDocIdSet for segment readers [LUCENE-2348]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
DuplicateFilter currently works by building a single doc ID set, without taking into account that getDocIdSet() will be called once per segment and only with each segment's local reader.
---
Migrated from [LUCENE-2348](https://issues.apache.org/jira/browse/LUCENE-2348) by Trejkaz, updated Nov 25 2014
Attachments: [LUCENE-2348.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-2348/LUCENE-2348.patch) (versions: 2)
Linked issues:
- #3580
Contributor guide
Research direction
Start by locating DuplicateFilter and its getDocIdSet() entry point, then trace how segment readers provide local document IDs. Compare the current single document ID set with the per-segment calls described in the issue and review linked issue #3580 or the attached LUCENE-2348.patch for context. Done means duplicate filtering works correctly across multiple segment readers.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100