IndexWriter.optimize() throws NullPointerException [LUCENE-2280]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
I am using lucene 2.3.2 search APIs for my application, i am indexing 45GB database which creates approax 200MB index file, after finishing the indexing and while running optimize() i can see NullPointerExcception thrown in my log and index file is getting corrupted, log says
-----
Caused by:
java.lang.NullPointerException
at org.apache.lucene.store.BufferedIndexOutput.writeBytes(BufferedIndexOutput.java:49)
at org.apache.lucene.store.IndexOutput.writeBytes(IndexOutput.java:40)
at org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:566)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:135)
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3273)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2968)
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:240)
-----
and this is happening quite frequently, although I am not able to reproduce it on demand, I saw an issue logged which is some what related to mine issue (http://mail-archives.apache.org/mod_mbox/lucene-solr-user/200809.mbox/%3C6E4A40DB-5EFC-42DA-A857-D59F4EC3496B@mikemccandless.com%3E) but the only difference here is I am not using Store.Compress for my fields, i am using Store.NO instead. please note that I am using IBM JRE for my application.
Is this an issue with lucene?, if yes it is fixed in which version?
---
Migrated from [LUCENE-2280](https://issues.apache.org/jira/browse/LUCENE-2280) by Ritesh Nigam, updated Nov 30 2013
Environment:
```
Win 2003, lucene version 2.3.2, IBM JRE 1.6
```
Attachments: [lucene.jar](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-2280/lucene.jar), [lucene.zip](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-2280/lucene.zip), [LuceneUtils.zip](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-2280/LuceneUtils.zip)
Contributor guide
Research direction
Start with the stack trace through BufferedIndexOutput, SegmentMerger.mergeNorms, and IndexWriter.optimize(), using the Lucene 2.3.2 environment details and attached archives. Compare the behavior with the related report cited in the issue and determine whether the failure can be reproduced. Done means optimize completes without the NullPointerException or index corruption.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100