apache / apache/lucene

Building a second index on a FAT32 drive from OSX fails with OverlappingFileLockException [LUCENE-3481]

Open
#4,555 3 comments 0 reactions 0 assignees View on GitHub
affects-version:3.1 legacy-jira-label:index legacy-jira-label:lock legacy-jira-priority:Major module:core/index type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

Building a new index using FSDirectory.open(new File(path) on a FAT32 drive from OSX. Once the index has built but the index writer has not been closed because I want to optimize it I create a new indexwriter to create a second completely different index. However this fails with

Exception in thread "main" java.nio.channels.OverlappingFileLockException
at sun.nio.ch.FileChannelImpl$SharedFileLockTable.checkList(FileChannelImpl.java:1166)
at sun.nio.ch.FileChannelImpl$SharedFileLockTable.add(FileChannelImpl.java:1068)
at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:868)
at java.nio.channels.FileChannel.tryLock(FileChannel.java:962)
at org.apache.lucene.store.NativeFSLock.obtain(NativeFSLockFactory.java:216)
at org.apache.lucene.store.Lock.obtain(Lock.java:72)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1097)

but this should not occur as I am not locking the same file, this second index is being built in a different directory albeit with the same parent directory as the other index.

I think the issue is specifically to do with OSX - FAT32 interaction because the problem always occurs, but never occurs if I build the indexes on the internal Mac drive.

The only way I could resolve this was to remove locking using

FSDirectory.open(new File(path), NoLockFactory.getNoLockFactory() );

---
Migrated from [LUCENE-3481](https://issues.apache.org/jira/browse/LUCENE-3481) by Paul taylor
Environment:
```
Mac OSX 10.7 Lion with attached FAT32 disc
```

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the failure with FSDirectory.open(new File(path)) on Mac OS X 10.7 using an attached FAT32 drive, while creating two indexes in separate directories sharing a parent. Read the NativeFSLock.obtain and IndexWriter stack frames first; done means the second IndexWriter can be created without OverlappingFileLockException while the indexes remain separate.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.