apache / apache/lucene

"Cannot commit index writer" in some cases on windows [LUCENE-7582]

Open
#8,633 7 comments 0 reactions 0 assignees View on GitHub
affects-version:5.3.1 legacy-jira-priority:Major module:core/store type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

Hi!

I've an error using lucene on windows. I already post a question on modeshape forum (https://developer.jboss.org/thread/273070) and it looks that NIOFSDirectory is not working well on windows as described in the java documentation of this class.

> NOTE: NIOFSDirectory is not recommended on Windows because of a bug in how FileChannel\.read is implemented in Sun's JRE\. Inside of the implementation the position is apparently synchronized\. See here for details\.

After reading the linked java issue (http://bugs.java.com/bugdatabase/view_bug.do?bug_id=6265734), it seems that there is a workaround to solve it, use an AsynchronousFileChannel.

Is it a choice that has been made to not use AsynchronousFileChannel or will it be a good fix?

You'll find the complete stacktrace below:

```java
Caused by: org.modeshape.jcr.index.lucene.LuceneIndexException: Cannot commit index writer
at org.modeshape.jcr.index.lucene.LuceneIndex.commit(LuceneIndex.java:155) \~[dsdk-launcher.jar:na]
at org.modeshape.jcr.spi.index.provider.IndexChangeAdapter.completeWorkspaceChanges(IndexChangeAdapter.java:104) \~[dsdk-launcher.jar:na]
at org.modeshape.jcr.cache.change.ChangeSetAdapter.notify(ChangeSetAdapter.java:157) \~[dsdk-launcher.jar:na]
at org.modeshape.jcr.spi.index.provider.IndexProvider$AtomicIndex.notify(IndexProvider.java:1493) \~[dsdk-launcher.jar:na]
at org.modeshape.jcr.bus.RepositoryChangeBus.notify(RepositoryChangeBus.java:190) \~[dsdk-launcher.jar:na]
at org.modeshape.jcr.cache.document.WorkspaceCache.changed(WorkspaceCache.java:333) \~[dsdk-launcher.jar:na]
at org.modeshape.jcr.txn.SynchronizedTransactions.updateCache(SynchronizedTransactions.java:223) \~[dsdk-launcher.jar:na]
at org.modeshape.jcr.cache.document.WritableSessionCache.save(WritableSessionCache.java:751) \~[dsdk-launcher.jar:na]
at org.modeshape.jcr.JcrSession.save(JcrSession.java:1171) \~[dsdk-launcher.jar:na]
... 19 common frames omitted
Caused by: java.nio.file.FileSystemException: C:\Users\Christopher\Infiltrea3CLOUDTEST8\christopher@dooapp.com\indexes\default\nodesByPath\_dc_Lucene50_0.doc: The process cannot access the file because it is being used by another process.
at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86) \~[na:1.8.0_92]
at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) \~[na:1.8.0_92]
at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102) \~[na:1.8.0_92]
at sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:115) \~[na:1.8.0_92]
at java.nio.channels.FileChannel.open(FileChannel.java:287) \~[na:1.8.0_92]
at java.nio.channels.FileChannel.open(FileChannel.java:335) \~[na:1.8.0_92]
at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:393) \~[dsdk-launcher.jar:na]
at org.apache.lucene.store.FSDirectory.fsync(FSDirectory.java:281) \~[dsdk-launcher.jar:na]
at org.apache.lucene.store.FSDirectory.sync(FSDirectory.java:226) \~[dsdk-launcher.jar:na]
at org.apache.lucene.store.LockValidatingDirectoryWrapper.sync(LockValidatingDirectoryWrapper.java:62) \~[dsdk-launcher.jar:na]
at org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4456) \~[dsdk-launcher.jar:na]
at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2874) \~[dsdk-launcher.jar:na]
at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2977) \~[dsdk-launcher.jar:na]
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2944) \~[dsdk-launcher.jar:na]
at org.modeshape.jcr.index.lucene.LuceneIndex.commit(LuceneIndex.java:152) \~[dsdk-launcher.jar:na]
```

Thank you in advance for your help

---
Migrated from [LUCENE-7582](https://issues.apache.org/jira/browse/LUCENE-7582) by Kevin Senechal, 1 vote, updated Dec 06 2016
Environment:
```
Windows 10, 32 bits JVM
```

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the Windows failure described in the stack trace, then read FSDirectory.fsync and sync alongside IndexWriter commit and the linked Java documentation and issue. Determine whether the failure is specific to Windows file-channel handling and define a supported resolution; the work is done when commits no longer produce the reported FileSystemException on the affected environment.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.