apache / apache/lucene

deleteAll() does not remove all TaxonomyWriter files [LUCENE-4884]

Open
#5,949 4 comments 0 reactions 0 assignees View on GitHub
affects-version:4.2 legacy-jira-priority:Minor module:core/index type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

When calling deleteAll() on the IndexWriter, the documents are removed from the index and from the taxonomy. When investigating what is happing after the deleteAll() on the disk, I see that in the index-directory I end up with just two files:

Index-directory:
- segments.gen
- segments_2

Taxonomy directory:
- segments.gen
- segments_h

BUT also a lot of 'older' files, like

- _1_Lucene41_0.tip
- _1_Lucene41_0.tim

etc.

It seems these files are never deleted. If you index a lot and call deleteAll a lot, it will fill up your disk.

---
Migrated from [LUCENE-4884](https://issues.apache.org/jira/browse/LUCENE-4884) by Rob Audenaerde

Contributor guide

Open the contributing guide

Research direction

Start at IndexWriter.deleteAll() and trace how deletion affects both the index-directory and taxonomy directory. Reproduce repeated deleteAll() calls while inspecting the directories, focusing on whether older taxonomy files such as _1_Lucene41_0.tip and _1_Lucene41_0.tim are removed. Done means obsolete files do not accumulate after documents and taxonomy entries are deleted.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.