FSDirectory can't open indexes that are symlinks, due to a deficiency in Files.createDirectories [LUCENE-6700]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
Lucene, using FSDirectory (via NIOFS directory) cannot open an index from a Path that is a symbolic link to an actual index directory. Trying to do so generates an exception stack like:
Exception in thread "main" java.nio.file.FileAlreadyExistsException: maildex.idx
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:88)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384)
at java.nio.file.Files.createDirectory(Files.java:674)
at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781)
at java.nio.file.Files.createDirectories(Files.java:727)
at org.apache.lucene.store.FSDirectory.<init>(FSDirectory.java:128)
at org.apache.lucene.store.NIOFSDirectory.<init>(NIOFSDirectory.java:64)
at org.apache.lucene.store.NIOFSDirectory.<init>(NIOFSDirectory.java:74)
This problem occurs on both Linux and Solaris (which probably use the same SPI for Unix file systems at the bottom of the java.nio.file stack.)
This problem has been noted in the OpenJDK issue tracker at:
https://bugs.openjdk.java.net/browse/JDK-8130464
And closed as "Not an Issue" because Files.createDirectories is meant to operate on directories, and a symlink is not a directory. This doesn't strike me as particularly helpful, but I guess is sort of makes sense in a broken-by-design way.
The work-around is simply to move or copy the index to the place where I want it, but this makes concurrent read-only development on the index difficult when the index is large.
---
Migrated from [LUCENE-6700](https://issues.apache.org/jira/browse/LUCENE-6700) by Stephen Green (@eelstretching), 1 vote
Environment:
```
java version "1.8.0_45"
Solaris or Linux
Symlinked directory
```
Contributor guide
Research direction
Start with the FSDirectory constructor and its NIOFSDirectory callers shown in the stack trace, then reproduce opening an index through a symbolic-link Path on Linux or Solaris. Done means a symlinked index directory opens without the reported FileAlreadyExistsException, while preserving normal directory behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 48/100