Random 7200 seconds build timeouts / infinite loops in Lucene tests? [LUCENE-6629]

I'm not sure what's going on here, but every so often a Jenkins run will fail with a build timeout (7200 seconds) with stack traces that do not look like deadlock. They never reproduce for me.

We really need to improve test infra here, so that each HEARTBEAT we also got 1) full thread stacks and 2) total CPU usage of the process as reported by the ManagementBean APIs ... this would shed more light on whether the JVM is somehow hung vs our bug (infinite loop). But infinite loop seems most likely ... the stacks always seem to be somewhere "spooky".

We should try to gather recent Jenkins runs where this is happening, here, to see if there are patterns / we can get to the root cause.

Anyway, this happened to me on my old beast box, which runs the "nightly ant test times" graphs: http://people.apache.org/\~mikemccand/lucenebench/antcleantest.html

The 2015/06/27 data point is missing because it failed with this timeout:

```
[junit4] Suite: org.apache.lucene.search.TestDocValuesRewriteMethod
[junit4] 2> ??? 28, 2015 7:01:29 ? com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.search.TestDocValuesRewriteMethod
[junit4] 2> 1) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1245)
[junit4] 2> at java.lang.Thread.join(Thread.java:1319)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:578)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:444)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:199)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:310)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
[junit4] 2> 2) Thread[id=213, name=TEST-TestDocValuesRewriteMethod.testRegexps-seed#[C2DDF486BB909D8], state=RUNNABLE, group=TGRP-TestDocValuesRewriteMethod]
[junit4] 2> at org.apache.lucene.util.automaton.Operations.getLiveStates(Operations.java:900)
[junit4] 2> at org.apache.lucene.util.automaton.Operations.hasDeadStates(Operations.java:389)
[junit4] 2> at org.apache.lucene.util.automaton.Automata.makeString(Automata.java:517)
[junit4] 2> at org.apache.lucene.util.automaton.RegExp.toAutomatonInternal(RegExp.java:579)
[junit4] 2> at org.apache.lucene.util.automaton.RegExp.findLeaves(RegExp.java:617)
[junit4] 2> at org.apache.lucene.util.automaton.RegExp.toAutomatonInternal(RegExp.java:519)
[junit4] 2> at org.apache.lucene.util.automaton.RegExp.findLeaves(RegExp.java:617)
[junit4] 2> at org.apache.lucene.util.automaton.RegExp.toAutomatonInternal(RegExp.java:510)
[junit4] 2> at org.apache.lucene.util.automaton.RegExp.toAutomaton(RegExp.java:495)
[junit4] 2> at org.apache.lucene.util.automaton.RegExp.toAutomaton(RegExp.java:466)
[junit4] 2> at org.apache.lucene.search.RegexpQuery.(RegexpQuery.java:109)
[junit4] 2> at org.apache.lucene.search.RegexpQuery.(RegexpQuery.java:79)
[junit4] 2> at org.apache.lucene.search.TestDocValuesRewriteMethod.assertSame(TestDocValuesRewriteMethod.java:117)
[junit4] 2> at org.apache.lucene.search.TestDocValuesRewriteMethod.testRegexps(TestDocValuesRewriteMethod.java:109)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:497)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 3) Thread[id=212, name=SUITE-TestDocValuesRewriteMethod-seed#[C2DDF486BB909D8], state=RUNNABLE, group=TGRP-TestDocValuesRewriteMethod]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1552)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:690)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:679)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$900(ThreadLeakControl.java:62)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:412)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:651)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:138)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$1.run(RandomizedRunner.java:568)
[junit4] 2> 4) Thread[id=19, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:47)
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestDocValuesRewriteMethod -Dtests.method=testRegexps -Dtests.seed=C2DDF486BB909D8 -Dtests.slow=true -Dtests.locale=ar_IQ -Dtests.timezone=Africa/Mbabane -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 7200s J6 | TestDocValuesRewriteMethod.testRegexps <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4] > at __randomizedtesting.SeedInfo.seed([C2DDF486BB909D8]:0)
[junit4] 2> ??? 28, 2015 7:01:29 ? com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> ??? 28, 2015 7:01:49 ? com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.search.TestDocValuesRewriteMethod:
[junit4] 2> 1) Thread[id=213, name=TEST-TestDocValuesRewriteMethod.testRegexps-seed#[C2DDF486BB909D8], state=RUNNABLE, group=TGRP-TestDocValuesRewriteMethod]
[junit4] 2> at org.apache.lucene.util.automaton.Operations.getLiveStates(Operations.java:900)
[junit4] 2> at org.apache.lucene.util.automaton.Operations.hasDeadStates(Operations.java:389)
[junit4] 2> at org.apache.lucene.util.automaton.Automata.makeString(Automata.java:517)
[junit4] 2> at org.apache.lucene.util.automaton.RegExp.toAutomatonInternal(RegExp.java:579)
[junit4] 2> at org.apache.lucene.util.automaton.RegExp.findLeaves(RegExp.java:617)
[junit4] 2> at org.apache.lucene.util.automaton.RegExp.toAutomatonInternal(RegExp.java:519)
[junit4] 2> at org.apache.lucene.util.automaton.RegExp.findLeaves(RegExp.java:617)
[junit4] 2> at org.apache.lucene.util.automaton.RegExp.toAutomatonInternal(RegExp.java:510)
[junit4] 2> at org.apache.lucene.util.automaton.RegExp.toAutomaton(RegExp.java:495)
[junit4] 2> at org.apache.lucene.util.automaton.RegExp.toAutomaton(RegExp.java:466)
[junit4] 2> at org.apache.lucene.search.RegexpQuery.(RegexpQuery.java:109)
[junit4] 2> at org.apache.lucene.search.RegexpQuery.(RegexpQuery.java:79)
[junit4] 2> at org.apache.lucene.search.TestDocValuesRewriteMethod.assertSame(TestDocValuesRewriteMethod.java:117)
[junit4] 2> at org.apache.lucene.search.TestDocValuesRewriteMethod.testRegexps(TestDocValuesRewriteMethod.java:109)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:497)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> ??? 28, 2015 7:01:49 ? com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=213, name=TEST-TestDocValuesRewriteMethod.testRegexps-seed#[C2DDF486BB909D8], state=RUNNABLE, group=TGRP-TestDocValuesRewriteMethod]
[junit4] 2> ??? 28, 2015 7:01:52 ? com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=213, name=TEST-TestDocValuesRewriteMethod.testRegexps-seed#[C2DDF486BB909D8], state=RUNNABLE, group=TGRP-TestDocValuesRewriteMethod]
[junit4] 2> at org.apache.lucene.util.automaton.Operations.getLiveStates(Operations.java:900)
[junit4] 2> at org.apache.lucene.util.automaton.Operations.hasDeadStates(Operations.java:389)
[junit4] 2> at org.apache.lucene.util.automaton.Automata.makeString(Automata.java:517)
[junit4] 2> at org.apache.lucene.util.automaton.RegExp.toAutomatonInternal(RegExp.java:579)
[junit4] 2> at org.apache.lucene.util.automaton.RegExp.findLeaves(RegExp.java:617)
[junit4] 2> at org.apache.lucene.util.automaton.RegExp.toAutomatonInternal(RegExp.java:519)
[junit4] 2> at org.apache.lucene.util.automaton.RegExp.findLeaves(RegExp.java:617)
[junit4] 2> at org.apache.lucene.util.automaton.RegExp.toAutomatonInternal(RegExp.java:510)
[junit4] 2> at org.apache.lucene.util.automaton.RegExp.toAutomaton(RegExp.java:495)
[junit4] 2> at org.apache.lucene.util.automaton.RegExp.toAutomaton(RegExp.java:466)
[junit4] 2> at org.apache.lucene.search.RegexpQuery.(RegexpQuery.java:109)
[junit4] 2> at org.apache.lucene.search.RegexpQuery.(RegexpQuery.java:79)
[junit4] 2> at org.apache.lucene.search.TestDocValuesRewriteMethod.assertSame(TestDocValuesRewriteMethod.java:117)
[junit4] 2> at org.apache.lucene.search.TestDocValuesRewriteMethod.testRegexps(TestDocValuesRewriteMethod.java:109)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:497)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene53): {=BlockTreeOrds(blocksize=128), id=PostingsFormat(name=Memory doPackFST= true)}, docValues:{=DocValuesFormat(name=Memory)}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=ar_IQ, timezone=Africa/Mbabane
[junit4] 2> NOTE: Linux 3.13.0-55-generic amd64/Oracle Corporation 1.8.0_40 (64-bit)/cpus=24,threads=2,free=270513952,total=324534272
[junit4] 2> NOTE: All tests run in this JVM: [TestBytesRefHash, TestRegexpQuery, TestIndexWriterUnicode, TestTopDocsMerge, TestUniqueTermCount, TestMinShouldMatch2, TestNot, TestSpanContainQuery, TestPayloadBasics, TestDoc, TestFastDecompressionMode, TestCachingCollector, TestSort, TestPriorityQueue, TestTwoPhaseCommitTool, TestCompiledAutomaton, TestSimpleFSDirectory, TestCloseableThreadLocal, TestIndexWriterExceptions, TestConcurrentMergeScheduler, TestSleepingLockWrapper, TestTieredMergePolicy, TestDeletionPolicy, TestFixedBitSet, TestDocValuesRewriteMethod]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestDocValuesRewriteMethod -Dtests.seed=C2DDF486BB909D8 -Dtests.slow=true -Dtests.locale=ar_IQ -Dtests.timezone=Africa/Mbabane -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J6 | TestDocValuesRewriteMethod (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([C2DDF486BB909D8]:0)
```

---
Migrated from [LUCENE-6629](https://issues.apache.org/jira/browse/LUCENE-6629) by Michael McCandless (@mikemccand), updated Dec 11 2015
Attachments: [54457_consoleText.txt](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-6629/54457_consoleText.txt)

Start by running the reported ant command for TestDocValuesRewriteMethod.testRegexps with seed C2DDF486BB909D8 and inspect the Operations and ThreadLeakControl stack entries. Review the test-heartbeat and ManagementBean context, then compare recent Jenkins failures to determine whether the timeout is a JVM hang or an infinite loop. Done means the failure pattern is understood and the requested diagnostic data can be collected.

Written by the indexing model from the issue text.