apache / apache/lucene

JVM 11.0.6 crash while trying to read term vectors in CheckIndex? [LUCENE-9987]

Open
#11,026 6 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

[This build](https://jenkins.thetaphi.de/job/Lucene-main-Linux/30482/) failed with JVM crash:
```
Current thread (0x00007f68780d24e0): JavaThread "TEST-TestExpressionSorts.testQueries-seed#[25E6600265A5C2D4]" [_thread_in_Java, id=530195, stack(0x00007f68ae1f2000,0x00007f68ae2f3000)]

Stack: [0x00007f68ae1f2000,0x00007f68ae2f3000], sp=0x00007f68ae2ef800, free space=1014k
Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
J 4987% c2 org.apache.lucene.codecs.lucene90.compressing.Lucene90CompressingTermVectorsReader.positionIndex(IILorg/apache/lucene/util/LongValues;[I)[[I (136 bytes) `@ 0x00007f69347af02e` [0x00007f69347ae480+0x0000000000000bae]
J 4952 c1 org.apache.lucene.codecs.lucene90.compressing.Lucene90CompressingTermVectorsReader.get(I)Lorg/apache/lucene/index/Fields; (2695 bytes) `@ 0x00007f692db07754` [0x00007f692db025c0+0x0000000000005194]
J 4895 c1 org.apache.lucene.codecs.asserting.AssertingTermVectorsFormat$AssertingTermVectorsReader.get(I)Lorg/apache/lucene/index/Fields; (26 bytes) `@ 0x00007f692daace2c` [0x00007f692daacd20+0x000000000000010c]
j org.apache.lucene.index.CheckIndex.testTermVectors(Lorg/apache/lucene/index/CodecReader;Ljava/io/PrintStream;ZZZ)Lorg/apache/lucene/index/CheckIndex$Status$TermVectorStatus;+96
j org.apache.lucene.index.CheckIndex.checkIndex(Ljava/util/List;)Lorg/apache/lucene/index/CheckIndex$Status;+1718
j org.apache.lucene.util.TestUtil.checkIndex(Lorg/apache/lucene/store/Directory;ZZLjava/io/ByteArrayOutputStream;)Lorg/apache/lucene/index/CheckIndex$Status;+67
j org.apache.lucene.store.MockDirectoryWrapper.close()V+276
j org.apache.lucene.expressions.TestExpressionSorts.tearDown()V+11
v \~StubRoutines::call_stub
V [libjvm.so+0x887569] JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, Thread*)+0x3b9
V [libjvm.so+0xcb1a2d] invoke(InstanceKlass*, methodHandle const&, Handle, bool, objArrayHandle, BasicType, objArrayHandle, bool, Thread*) [clone .constprop.80]+0x43d
V [libjvm.so+0xcb2a62] Reflection::invoke_method(oopDesc*, Handle, objArrayHandle, Thread*)+0x102
V [libjvm.so+0x93b0cc] JVM_InvokeMethod+0xfc
j jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Ljava/lang/reflect/Method;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+0 java.base@11.0.6
j jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+100 java.base@11.0.6
j jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+6 java.base@11.0.6
j java.lang.reflect.Method.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+59 java.base@11.0.6
j com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(Ljava/lang/reflect/Method;Ljava/lang/Object;[Ljava/lang/Object;)V+69
j com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate()V+69
j org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate()V+20
```

---
Migrated from [LUCENE-9987](https://issues.apache.org/jira/browse/LUCENE-9987) by Michael McCandless (@mikemccand)
Attachments: [hs_err_pid529873.log](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-9987/hs_err_pid529873.log), [hs_err_pid536810.log](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-9987/hs_err_pid536810.log)

Contributor guide

Open the contributing guide

Research direction

Start with the JVM crash logs and the stack frames for Lucene90CompressingTermVectorsReader.positionIndex/get, then trace how CheckIndex.testTermVectors reaches them through TestExpressionSorts. Reproduce the failure on JVM 11.0.6 if possible and determine what change prevents the crash while preserving term-vector checking.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.