IndexReader.reopen on SSHFS mounted index eventually fails with read past EOF [LUCENE-2538]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
I have an index that Solr manages (as the sole known writer). I have mounted it over sshfs. Periodically calling IndexReader.reopen() on that index will result in the following error:
[java] java.io.IOException: read past EOF
[java] at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:151)
[java] at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:38)
[java] at org.apache.lucene.store.IndexInput.readInt(IndexInput.java:68)
[java] at org.apache.lucene.index.FieldsReader.<init>(FieldsReader.java:110)
[java] at org.apache.lucene.index.SegmentReader$CoreReaders.openDocStores(SegmentReader.java:291)
[java] at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:654)
[java] at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:613)
[java] at org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:228)
[java] at org.apache.lucene.index.ReadOnlyDirectoryReader.<init>(ReadOnlyDirectoryReader.java:32)
[java] at org.apache.lucene.index.DirectoryReader.doReopen(DirectoryReader.java:440)
[java] at org.apache.lucene.index.DirectoryReader.access$000(DirectoryReader.java:43)
[java] at org.apache.lucene.index.DirectoryReader$2.doBody(DirectoryReader.java:432)
[java] at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:683)
[java] at org.apache.lucene.index.DirectoryReader.doReopenNoWriter(DirectoryReader.java:428)
[java] at org.apache.lucene.index.DirectoryReader.doReopen(DirectoryReader.java:386)
[java] at org.apache.lucene.index.DirectoryReader.reopen(DirectoryReader.java:352)
I've read in other similar bugs that this indicates a corrupt index. I have had no problems with the index so far and solr seems be fine with it, so I'm not sure if that's the case. I know mounting an index over sshfs is not recommended, and I have since moved to solr replication, but I wanted to report this in case someone else encounters it.
---
Migrated from [LUCENE-2538](https://issues.apache.org/jira/browse/LUCENE-2538) by Max Lynch
Environment:
```
Linux 2.6.31-302-rs #7 SMP x86_64
Ubuntu 9.10
Java(TM) SE Runtime Environment (build 1.6.0_20-b02)
```
Contributor guide
Research direction
Start with DirectoryReader.reopen, SegmentInfos.FindSegmentsFile, and the BufferedIndexInput, FieldsReader, and SegmentReader locations named in the stack trace. Reproduce periodic reopen calls against an SSHFS-mounted index and determine whether the failure is caused by remote filesystem behavior or index corruption. The report provides no target file, test, or acceptance condition, so the definition of done remains unresolved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100