ArrayIndexOutOfBoundsException in reading a lucene block [LUCENE-6327]

I notice this error while trying to do heavy indexing in Solr. This error was seen in testing Cloudera Search (Solr 4.4 with lots of SolrCloud, other critical bug fixes)

```Java
2015-02-27 04:21:46,644 INFO org.apache.solr.core.SolrCore.Request: [crunch_sequence_collection_shard2_replica5] webapp=/solr path=/update params={distrib.from=http://search-15.vpc.cloudera.com:8983/solr/crunch_sequence_collection_shard2_replica2/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=246
2015-02-27 04:21:46,773 ERROR org.apache.solr.core.SolrCore: java.lang.ArrayIndexOutOfBoundsException
at org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum$Frame.fillTerm(BlockTreeTermsReader.java:2934)
at org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum$Frame.scanToTermLeaf(BlockTreeTermsReader.java:2743)
at org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum$Frame.scanToTerm(BlockTreeTermsReader.java:2662)
at org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum.seekExact(BlockTreeTermsReader.java:1695)
at org.apache.solr.search.SolrIndexSearcher.lookupId(SolrIndexSearcher.java:746)
at org.apache.solr.update.VersionInfo.getVersionFromIndex(VersionInfo.java:193)
at org.apache.solr.update.UpdateLog.lookupVersion(UpdateLog.java:739)
at org.apache.solr.update.VersionInfo.lookupVersion(VersionInfo.java:183)
at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:716)
at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:459)
at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:247)
at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:174)
at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1953)
at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:766)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:397)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:186)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.solr.servlet.SolrHadoopAuthenticationFilter$2.doFilter(SolrHadoopAuthenticationFilter.java:272)
at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:592)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.doFilter(DelegationTokenAuthenticationFilter.java:277)
at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:555)
at org.apache.solr.servlet.SolrHadoopAuthenticationFilter.doFilter(SolrHadoopAuthenticationFilter.java:277)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.solr.servlet.HostnameFilter.doFilter(HostnameFilter.java:86)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:606)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:745)

2015-02-27 04:21:46,773 INFO org.apache.solr.core.SolrCore.Request: [crunch_sequence_collection_shard2_replica5] webapp=/solr path=/update params={distrib.from=http://search-15.vpc.cloudera.com:8983/solr/crunch_sequence_collection_shard2_replica2/&update.distrib=FROMLEADER&wt=javabin&version=2} status=500 QTime=127
2015-02-27 04:21:46,792 ERROR org.apache.solr.servlet.SolrDispatchFilter: null:java.lang.ArrayIndexOutOfBoundsException
at org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum$Frame.fillTerm(BlockTreeTermsReader.java:2934)
at org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum$Frame.scanToTermLeaf(BlockTreeTermsReader.java:2743)
at org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum$Frame.scanToTerm(BlockTreeTermsReader.java:2662)
at org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum.seekExact(BlockTreeTermsReader.java:1695)
at org.apache.solr.search.SolrIndexSearcher.lookupId(SolrIndexSearcher.java:746)
at org.apache.solr.update.VersionInfo.getVersionFromIndex(VersionInfo.java:193)
at org.apache.solr.update.UpdateLog.lookupVersion(UpdateLog.java:739)
at org.apache.solr.update.VersionInfo.lookupVersion(VersionInfo.java:183)
at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:716)
at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:459)
at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:247)
at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:174)
at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1953)
at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:766)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:397)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:186)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.solr.servlet.SolrHadoopAuthenticationFilter$2.doFilter(SolrHadoopAuthenticationFilter.java:272)
at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:592)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.doFilter(DelegationTokenAuthenticationFilter.java:277)
at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:555)
at org.apache.solr.servlet.SolrHadoopAuthenticationFilter.doFilter(SolrHadoopAuthenticationFilter.java:277)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.solr.servlet.HostnameFilter.doFilter(HostnameFilter.java:86)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:606)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:745)
```

This error was seen on one of the solr server logs which is lagging behind in updates. Overall this looks more like a low level lucene issue as no matter what the Solr input is, lucene should handle all possible end cases right.

---
Migrated from [LUCENE-6327](https://issues.apache.org/jira/browse/LUCENE-6327) by Vamsee K. Yarlagadda, 2 votes, updated Oct 15 2018

Start in org.apache.lucene.codecs.BlockTreeTermsReader.java around the reported fillTerm, scanToTermLeaf, scanToTerm, and seekExact stack frames, then trace the SolrIndexSearcher.lookupId entry point. Reproduce the failure during heavy indexing or lagging SolrCloud updates and determine the conditions that trigger it; done means the reported exception no longer occurs and the regression is covered.

Written by the indexing model from the issue text.