apache / apache/lucene

TestIndexWriterReader fails: too many open files [LUCENE-2873]

Open
#3,947 2 comments 0 reactions 0 assignees View on GitHub
affects-version:3.1 legacy-jira-priority:Major type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

```
[junit] Testsuite: org.apache.lucene.index.TestIndexWriterReader
[junit] Testcase: testAddIndexesAndDoDeletesThreads(org.apache.lucene.index.TestIndexWriterReader): Caused an ERROR
[junit] /home/cron/branch_3x/lucene/build/test/6/test7430286492423218781tmp/_90.prx (Too many open files)
[junit] java.io.FileNotFoundException: /home/cron/branch_3x/lucene/build/test/6/test7430286492423218781tmp/_90.prx (Too many open files)
[junit] at java.io.RandomAccessFile.open(Native Method)
[junit] at java.io.RandomAccessFile.(RandomAccessFile.java:229)
[junit] at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput$Descriptor.(SimpleFSDirectory.java:69)
[junit] at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.(SimpleFSDirectory.java:90)
[junit] at org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.(NIOFSDirectory.java:91)
[junit] at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:78)
[junit] at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:353)
[junit] at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:358)
[junit] at org.apache.lucene.store.Directory.openInput(Directory.java:139)
[junit] at org.apache.lucene.index.SegmentReader$CoreReaders.(SegmentReader.java:135)
[junit] at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:583)
[junit] at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:561)
[junit] at org.apache.lucene.index.DirectoryReader.(DirectoryReader.java:101)
[junit] at org.apache.lucene.index.ReadOnlyDirectoryReader.(ReadOnlyDirectoryReader.java:27)
[junit] at org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:78)
[junit] at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:697)
[junit] at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:72)
[junit] at org.apache.lucene.index.IndexReader.open(IndexReader.java:344)
[junit] at org.apache.lucene.index.IndexReader.open(IndexReader.java:230)
[junit] at org.apache.lucene.index.TestIndexWriterReader.testAddIndexesAndDoDeletesThreads(TestIndexWriterReader.java:381)
[junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1007)
[junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:939)
[junit]
[junit]
[junit] Tests run: 18, Failures: 0, Errors: 1, Time elapsed: 13.56 sec
[junit]
[junit] ------------- Standard Error -----------------
[junit] NOTE: reproduce with: ant test -Dtestcase=TestIndexWriterReader -Dtestmethod=testAddIndexesAndDoDeletesThreads -Dtests.seed=-7781539944268912038:-6865031686554264582
[junit] NOTE: test params are: locale=ar_SD, timezone=Asia/Almaty
[junit] NOTE: all tests run in this JVM:
[junit] [TestMergeSchedulerExternal, TestCharFilter, TestISOLatin1AccentFilter, TestCharTermAttributeImpl, TestDoc, TestFieldsReader, TestFilterIndexReader, TestIndexWriterReader]
[junit] NOTE: Linux 2.6.32-24-generic x86/IBM Corporation 1.6.0 (32-bit)/cpus=1,threads=3,free=6495816,total=11920384
[junit] ------------- ---------------- ---------------
[junit] TEST org.apache.lucene.index.TestIndexWriterReader FAILED
```

---
Migrated from [LUCENE-2873](https://issues.apache.org/jira/browse/LUCENE-2873) by Robert Muir (@rmuir), updated Oct 14 2015
Environment:
```
java version "1.6.0"
Java(TM) SE Runtime Environment (build pxi3260sr9-20101125_01(SR9))
IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux x86-32 jvmxi3260sr9-20101124_69295 (JIT enabled, AOT enabled)
J9VM - 20101124_069295
JIT - r9_20101028_17488ifx2
GC - 20101027_AA)
JCL - 20101119_01

```

Contributor guide

Open the contributing guide

Research direction

Start by reproducing org.apache.lucene.index.TestIndexWriterReader.testAddIndexesAndDoDeletesThreads with ant test -Dtestcase=TestIndexWriterReader -Dtestmethod=testAddIndexesAndDoDeletesThreads and the recorded seed. Trace the open-input path through NIOFSDirectory, MockDirectoryWrapper, SegmentReader, and DirectoryReader; done means the test no longer errors with Too many open files.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.