apache / apache/lucene

Fix MockDirectoryWrapper double-write logic? [LUCENE-5903]

Open
#6,965 2 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

From a jenkins failure:

> I think its a bug, and this was just exposed by randomization of
> commitOnClose in IndexWriterConfig?
>
> we start a CMS merge for _8 segment (consumer creates _8\.fdt and
> _8\.fdx), but this merge doesn't make it into the commit, and we
> close() without waiting for merges\.
> this merge is aborted and indexfiledeleter cleans up _8\.fdt and _8\.fdx
>
> we start a new IW on the same dir, and the next segment it tries to
> create is _8 (how else would it know?) which trips the double-write
> logic in createOutput\.
> This logic currently only checks files that we ever created, and
> doesn't check if we ever deleted the file\.
> We can fix mockdirectorywrapper, but I feel like this could be a real
> problem on e\.g\. windows if the file was busy for both IFD runs\.

---
Migrated from [LUCENE-5903](https://issues.apache.org/jira/browse/LUCENE-5903) by Robert Muir (@rmuir)
Attachments: [LUCENE-5903_test.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-5903/LUCENE-5903_test.patch)

Contributor guide

Open the contributing guide

Research direction

Start by reviewing MockDirectoryWrapper and its createOutput double-write tracking, then read the attached LUCENE-5903_test.patch. Reproduce the sequence where a merge creates and deletes _8 files before a new IndexWriter reuses the name. Done means the test captures the scenario and deleted files no longer cause a false double-write failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.