Hang in Lucene trunk while indexing [LUCENE-4151]

See: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java7-64/309/

I stopped the build, 2 test childs were still running 100% CPU in analysis tests. I was ablte to get stack trace of one of then, see build log:

```
[junit4] "TEST-TestScope-org.apache.lucene.analysis.hunspell.HunspellStemFilterTest.testRandomStrings-seed#[BEEAD57FE445FA49]" prio=10 tid=0x00007fde184d8800 nid=0x497d runnable [0x00007fde0bd7a000]
[junit4] java.lang.Thread.State: RUNNABLE
[junit4] at org.apache.lucene.codecs.lucene40.values.Bytes$DerefBytesWriterBase.fillDefault(Bytes.java:456)
[junit4] at org.apache.lucene.codecs.lucene40.values.Bytes$DerefBytesWriterBase.add(Bytes.java:435)
[junit4] at org.apache.lucene.codecs.lucene40.values.FixedDerefBytesImpl$Writer.add(FixedDerefBytesImpl.java:48)
[junit4] at org.apache.lucene.codecs.DocValuesConsumer.mergeDoc(DocValuesConsumer.java:237)
[junit4] at org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:184)
[junit4] at org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:112)
[junit4] at org.apache.lucene.codecs.PerDocConsumer.merge(PerDocConsumer.java:60)
[junit4] at org.apache.lucene.index.SegmentMerger.mergePerDoc(SegmentMerger.java:351)
[junit4] at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:119)
[junit4] at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3477)
[junit4] at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3112)
[junit4] at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:37)
[junit4] - locked <0x00000000f8cb6320> (a org.apache.lucene.index.SerialMergeScheduler)
[junit4] at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1706)
[junit4] at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1700)
[junit4] at org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1197)
[junit4] at org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1147)
[junit4] at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:157)
[junit4] at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:145)
[junit4] at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:558)
[junit4] at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:446)
[junit4] at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:373)
[junit4] at org.apache.lucene.analysis.hunspell.HunspellStemFilterTest.testRandomStrings(HunspellStemFilterTest.java:75)
[junit4] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] at java.lang.reflect.Method.invoke(Method.java:601)
[junit4] at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969)
[junit4] at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
[junit4] at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814)
[junit4] at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875)
[junit4] at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889)
[junit4] at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4] at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
[junit4] at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
[junit4] at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:821)
[junit4] at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)
[junit4] at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:669)
[junit4] at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:695)
[junit4] at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:734)
[junit4] at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:745)
[junit4] at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
[junit4] at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
[junit4] at org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51)
[junit4] at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] at org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53)
[junit4] at org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52)
[junit4] at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36)
[junit4] at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:56)
[junit4] at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605)
[junit4] at com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132)
[junit4] at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551)
```

Please note the line with "locked in SerialMergeScheduler"!

---
Migrated from [LUCENE-4151](https://issues.apache.org/jira/browse/LUCENE-4151) by Uwe Schindler (@uschindler), updated May 09 2016

Start with HunspellStemFilterTest.testRandomStrings and the reported stack trace, then inspect Bytes.java, DocValuesConsumer.merge, and SerialMergeScheduler. Reproduce the Jenkins hang if possible and determine why indexing remains at 100% CPU while SerialMergeScheduler is locked. Done means the test completes without the reported hang.

Written by the indexing model from the issue text.