apache / apache/lucene

AnalyzingSuggester exception because of length restriction: java.lang.IllegalArgumentException: len must be <= 32767; got 38751 [LUCENE-6012]

Open
#7,074 8 comments 0 reactions 0 assignees View on GitHub
affects-version:4.10.1 legacy-jira-label:length legacy-jira-label:short legacy-jira-label:suggest legacy-jira-priority:Major type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

I want to use a suggester but every time i want to build the index I get an exception.
As the exception comes from AnalyzingSuggester all kind of suggesters that are subclasses fail to build.
By now I don't understand the code for the suggester well and thus was not able to track the error further down.

OfflineSorter$ByteSequencesWriter tries to write the length of the bytearray which not exceed 32767 because it is a short.

6877 ERROR org.apache.solr.core.SolrCore - java.lang.IllegalArgumentException: len must be <= 32767; got 38751
at org.apache.lucene.util.OfflineSorter$ByteSequencesWriter.write(OfflineSorter.java:479)
at org.apache.lucene.search.suggest.analyzing.AnalyzingSuggester.build(AnalyzingSuggester.java:496)
at org.apache.lucene.search.suggest.Lookup.build(Lookup.java:190)

---
Migrated from [LUCENE-6012](https://issues.apache.org/jira/browse/LUCENE-6012) by Daniel Aschauer, updated Jul 24 2015

Contributor guide

Open the contributing guide

Research direction

Start with OfflineSorter$ByteSequencesWriter.write at OfflineSorter.java:479 and the call from AnalyzingSuggester.build at AnalyzingSuggester.java:496. Reproduce the reported suggester build failure with an input producing a byte array of length 38751, then trace how the length restriction is reached. Done means the reported build no longer fails because of this restriction.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.