apache / apache/lucene

FSDirectory can't open indexes that are symlinks, due to a deficiency in Files.createDirectories [LUCENE-6700]

Open
#7,758 0 comments 0 reactions 0 assignees View on GitHub
affects-version:5.0 legacy-jira-priority:Minor module:core/index type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

Lucene, using FSDirectory (via NIOFS directory) cannot open an index from a Path that is a symbolic link to an actual index directory. Trying to do so generates an exception stack like:

Exception in thread "main" java.nio.file.FileAlreadyExistsException: maildex.idx
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:88)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384)
at java.nio.file.Files.createDirectory(Files.java:674)
at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781)
at java.nio.file.Files.createDirectories(Files.java:727)
at org.apache.lucene.store.FSDirectory.<init>(FSDirectory.java:128)
at org.apache.lucene.store.NIOFSDirectory.<init>(NIOFSDirectory.java:64)
at org.apache.lucene.store.NIOFSDirectory.<init>(NIOFSDirectory.java:74)

This problem occurs on both Linux and Solaris (which probably use the same SPI for Unix file systems at the bottom of the java.nio.file stack.)

This problem has been noted in the OpenJDK issue tracker at:

https://bugs.openjdk.java.net/browse/JDK-8130464

And closed as "Not an Issue" because Files.createDirectories is meant to operate on directories, and a symlink is not a directory. This doesn't strike me as particularly helpful, but I guess is sort of makes sense in a broken-by-design way.

The work-around is simply to move or copy the index to the place where I want it, but this makes concurrent read-only development on the index difficult when the index is large.

---
Migrated from [LUCENE-6700](https://issues.apache.org/jira/browse/LUCENE-6700) by Stephen Green (@eelstretching), 1 vote
Environment:
```
java version "1.8.0_45"
Solaris or Linux
Symlinked directory
```

Contributor guide

Open the contributing guide

Research direction

Start with the FSDirectory constructor and its NIOFSDirectory callers shown in the stack trace, then reproduce opening an index through a symbolic-link Path on Linux or Solaris. Done means a symlinked index directory opens without the reported FileAlreadyExistsException, while preserving normal directory behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.