apache / apache/lucene

TestLongBitSet.testHugeCapacity sometimes fails on nightly [LUCENE-9419]

Open
#10,459 0 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Minor type:test
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

I see this once in a while on our CI with -Dtests.nightly=true
```
00:05:30 [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestLongBitSet -Dtests.method=testHugeCapacity -Dtests.seed=FB56CEB40315E44E -Dtests.nightly=true -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=lu -Dtests.timezone=Africa/Casablanca -Dtests.asserts=true -Dtests.file.encoding=UTF-8
00:05:30 [junit4] ERROR 0.08s J1 | TestLongBitSet.testHugeCapacity <<<
00:05:30 [junit4] > Throwable #1: java.lang.OutOfMemoryError: Java heap space
00:05:30 [junit4] > at __randomizedtesting.SeedInfo.seed([FB56CEB40315E44E:8116844FEAE5C24A]:0)
00:05:30 [junit4] > at org.apache.lucene.util.ArrayUtil.growExact(ArrayUtil.java:323)
00:05:30 [junit4] > at org.apache.lucene.util.ArrayUtil.grow(ArrayUtil.java:332)
00:05:30 [junit4] > at org.apache.lucene.util.LongBitSet.ensureCapacity(LongBitSet.java:54)
00:05:30 [junit4] > at org.apache.lucene.util.TestLongBitSet.testHugeCapacity(TestLongBitSet.java:343)
00:05:30 [junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
00:05:30 [junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
00:05:30 [junit4] > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
00:05:30 [junit4] > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
00:05:30 [junit4] 2> NOTE: test params are: codec=Asserting(Lucene86): {}, docValues:{}, maxPointsInLeafNode=246, maxMBSortInHeap=6.097958740443044, sim=Asserting(RandomSimilarity(queryNorm=false): {}), locale=lu, timezone=Africa/Casablanca
00:05:30 [junit4] 2> NOTE: Linux 4.12.14-122.26-default amd64/Oracle Corporation 11.0.2 (64-bit)/cpus=32,threads=1,free=346124224,total=362807296
00:05:30 [junit4] 2> NOTE: All tests run in this JVM: [TestSynonymQuery, TestCachingCollector, TestLucene86PointsFormat, TestSizeBoundedForceMerge, TestSimpleWKTShapeParsing, TestMergeRateLimiter, TestAxiomaticSimilarity, TestReaderClosed, TestSpanExplanations, TestSingleInstanceLockFactory, TestIndexSorting, TestIndexInput, TestBasicModelIn, TestPoint2D, TestAtomicUpdate, TestOneMergeWrappingMergePolicy, TestTragicIndexWriterDeadlock, TestRollingUpdates, TestCodecHoldsOpenFiles, TestLucene50StoredFieldsFormatHighCompression, TestFieldsReader, TestAxiomaticF2EXP, TestPointValues, TestIndependenceStandardized, TestTwoPhaseCommitTool, TestLongBitSet]
00:05:30 [junit4] Completed [87/566 (1!)] on J1 in 3.30s, 11 tests, 1 error <<< FAILURES!
```

my suspicion is that some other test is holding on to memory here. I am trying to record all failures here maybe we find some tests in common. The test itself needs about 300MB to run so that should be ok with a 512M JVM

---
Migrated from [LUCENE-9419](https://issues.apache.org/jira/browse/LUCENE-9419) by Simon Willnauer (@s1monw)

Contributor guide

Open the contributing guide

Research direction

Reproduce TestLongBitSet.testHugeCapacity with the provided ant command and seed, then inspect TestLongBitSet.java:343, LongBitSet.ensureCapacity, and ArrayUtil.growExact. Determine why the nightly test reaches an OutOfMemoryError despite the 512M JVM and verify that the test no longer fails intermittently under the reported conditions.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.