Search Index Generation fails [LUCENE-2800]

Hi,

We are using lucene 2.0.0 for search index In our Comergent application
It was working fine since from more than 3 years.
From this week, it is throwing Exception while creating New Index and also for Incremental Index.
Below is the exception

com.comergent.api.appservices.productService.ProductServiceException: java.io.IOException: Cannot delete ...\searchIndex\en_US\MasterIndex_602580\segments
at com.comergent.reference.appservices.productService.search.indexBuilder.CatalogIndexSetBuilder.indexPCFromCache(CatalogIndexSetBuilder.java:634)
at com.comergent.reference.appservices.productService.search.indexBuilder.CatalogIndexSetBuilder.buildIndexSet(CatalogIndexSetBuilder.java:276)
at com.comergent.appservices.search.indexBuilder.IndexSetBuilder$BuilderThread.run(IndexSetBuilder.java:469)
Caused by: java.io.IOException: Cannot delete ----searchIndex\en_US\MasterIndex_602580\segments
at org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:268)
at org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:95)
at org.apache.lucene.index.IndexWriter$4.doBody(IndexWriter.java:726)
at org.apache.lucene.store.Lock$With.run(Lock.java:99)
at org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:724)
at org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:686)
at org.apache.lucene.index.IndexWriter.maybeMergeSegments(IndexWriter.java:674)
at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:479)
at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:462)
at com.comergent.reference.appservices.productService.search.indexBuilder.CatalogIndexSetBuilder.indexPCFromCache(CatalogIndexSetBuilder.java:630)
... 2 more
2010.12.05 06:25:13:532 Env/Thread-21961:ERROR:CatalogIndexSetBuilder CatalogIndexSetBuilder: [MasterIndex_602580] - Exception: com.comergent.api.appservices.productService.ProductServiceException: java.io.IOException: Cannot delete ...\MasterIndex_602580\segments
2010.12.05 06:25:13:532 Env/Thread-21961:INFO:CMGT_SEARCH IndexSetBuilder$BuilderThread: error building the index for: MasterIndex_602580
com.comergent.api.exception.ComergentException: com.comergent.api.appservices.productService.ProductServiceException: java.io.IOException: Cannot delete \searchIndex\en_US\MasterIndex_602580\segments
at com.comergent.reference.appservices.productService.search.indexBuilder.CatalogIndexSetBuilder.buildIndexSet(CatalogIndexSetBuilder.java:305)
at com.comergent.appservices.search.indexBuilder.IndexSetBuilder$BuilderThread.run(IndexSetBuilder.java:469)
Caused by: com.comergent.api.appservices.productService.ProductServiceException: java.io.IOException: Cannot delete ...\MasterIndex_602580\segments
at com.comergent.reference.appservices.productService.search.indexBuilder.CatalogIndexSetBuilder.indexPCFromCache(CatalogIndexSetBuilder.java:634)
at com.comergent.reference.appservices.productService.search.indexBuilder.CatalogIndexSetBuilder.buildIndexSet(CatalogIndexSetBuilder.java:276)
... 1 more
Caused by: java.io.IOException: Cannot delete ...\MasterIndex_602580\segments
at org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:268)
at org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:95)
at org.apache.lucene.index.IndexWriter$4.doBody(IndexWriter.java:726)
at org.apache.lucene.store.Lock$With.run(Lock.java:99)
at org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:724)
at org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:686)
at org.apache.lucene.index.IndexWriter.maybeMergeSegments(IndexWriter.java:674)
at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:479)
at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:462)
at com.comergent.reference.appservices.productService.search.indexBuilder.CatalogIndexSetBuilder.indexPCFromCache(CatalogIndexSetBuilder.java:630)
... 2 more

2010.12.05 06:25:13:938 Env/http-8080-Processor75:INFO:CMGT_SEARCH IndexSetBuilder: error building the index: com.comergent.api.appservices.search.exception.IndexingException: Error in executing some builder threads...
at com.comergent.appservices.search.indexBuilder.IndexSetBuilder.monitor(IndexSetBuilder.java:440)
at com.comergent.appservices.search.indexBuilder.IndexSetBuilder.build(IndexSetBuilder.java:185)
at com.comergent.reference.appservices.search.indexBuilder.ComergentIndexSetManager.startIndexBuilder(ComergentIndexSetManager.java:269)
at com.comergent.appservices.search.indexBuilder.IndexSetManager.startFullIndexBuilder(IndexSetManager.java:516)
at com.comergent.reference.appservices.search.indexBuilder.IndexMaintenanceBLC.service(IndexMaintenanceBLC.java:145)
at com.comergent.dcm.core.AppExecutionEnv.runAppObj(AppExecutionEnv.java:77)
at com.comergent.dcm.messaging.MessagingController.handleRequest(MessagingController.java:199)
at com.comergent.dcm.messaging.MessagingController.execute(MessagingController.java:103)
at com.comergent.dcm.core.DispatchServlet.executeController(DispatchServlet.java:485)
at com.comergent.dcm.core.DispatchServlet.doExecute(DispatchServlet.java:434)
at com.comergent.dcm.messaging.MessagingServlet.execute(MessagingServlet.java:161)
at com.comergent.dcm.core.DispatchServlet.dispatch(DispatchServlet.java:189)
at com.comergent.dcm.core.DispatchServlet.doPost(DispatchServlet.java:157)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at com.comergent.dcm.core.filters.ComergentFilter.doFilter(ComergentFilter.java:50)
at com.comergent.dcm.core.filters.CredentialPropagationFilter.executeFilter(CredentialPropagationFilter.java:57)
at com.comergent.dcm.core.filters.ComergentFilter.doFilter(ComergentFilter.java:58)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at com.comergent.dcm.core.filters.ComergentFilter.doFilter(ComergentFilter.java:50)
at com.comergent.dcm.core.filters.AAFilter.executeFilter(AAFilter.java:54)
at com.comergent.dcm.core.filters.ComergentFilter.doFilter(ComergentFilter.java:58)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at com.comergent.dcm.core.filters.ComergentFilter.doFilter(ComergentFilter.java:50)
at com.comergent.dcm.core.WrappingFilter.executeFilter(WrappingFilter.java:113)
at com.comergent.dcm.core.filters.ComergentFilter.doFilter(ComergentFilter.java:58)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at com.comergent.dcm.core.filters.ComergentFilter.doFilter(ComergentFilter.java:50)
at com.comergent.dcm.core.filters.StatsFilter.executeFilter(StatsFilter.java:38)
at com.comergent.dcm.core.filters.ComergentFilter.doFilter(ComergentFilter.java:58)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at com.comergent.dcm.core.filters.SSLSessionKeyFilter.handleHttpRequest(SSLSessionKeyFilter.java:79)
at com.comergent.dcm.core.filters.SSLSessionKeyFilter.doFilter(SSLSessionKeyFilter.java:186)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at com.comergent.dcm.core.filters.IPSessionTrackingFilter.doFilter(IPSessionTrackingFilter.java:67)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at com.comergent.dcm.core.filters.ComergentFilter.doFilter(ComergentFilter.java:50)
at com.comergent.dcm.core.filters.RequestControlFilter.executeFilter(RequestControlFilter.java:102)
at com.comergent.dcm.core.filters.ComergentFilter.doFilter(ComergentFilter.java:58)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at com.comergent.dcm.core.filters.ComergentFilter.doFilter(ComergentFilter.java:50)
at com.comergent.dcm.core.filters.TimingFilter.executeFilter(TimingFilter.java:46)
at com.comergent.dcm.core.filters.ComergentFilter.doFilter(ComergentFilter.java:58)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at com.comergent.dcm.core.filters.ComergentFilter.doFilter(ComergentFilter.java:50)
at com.comergent.dcm.core.filters.EntryFilter.executeFilter(EntryFilter.java:51)
at com.comergent.dcm.core.filters.ComergentFilter.doFilter(ComergentFilter.java:58)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
at org.jstripe.tomcat.probe.Tomcat55AgentValve.invoke(Tomcat55AgentValve.java:20)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
at java.lang.Thread.run(Unknown Source)

Could you please help us to solve this issue.

---
Migrated from [LUCENE-2800](https://issues.apache.org/jira/browse/LUCENE-2800) by Sunitha Belavagi, updated Nov 30 2013
Environment:
```
Windows Server 2003
```

Start by reproducing the failure on the reported Windows Server 2003 environment and trace the stack from CatalogIndexSetBuilder.indexPCFromCache through IndexWriter.addDocument to FSDirectory.renameFile. Check the involved Lucene 2.0.0 classes, especially SegmentInfos.write, to determine why the segments file cannot be deleted. Done means both new and incremental index generation complete successfully.

Written by the indexing model from the issue text.