CompressingStoredFieldsWriter on too-big document [LUCENE-5845]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
This guy has a documented limit of 2^31-2^14
But it becomes possible (with LUCENE-5844) to add a document that exceeds this... we shouldn't give AIOOBE but something more clear than this:
```
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestDemo -Dtests.method=testMassiveDoc -Dtests.seed=8306F98D2E2B9750 -Dtests.locale=pl -Dtests.timezone=America/Jamaica -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 5.76s | TestDemo.testMassiveDoc <<<
[junit4] > Throwable #1: java.lang.ArrayIndexOutOfBoundsException
[junit4] > at __randomizedtesting.SeedInfo.seed([8306F98D2E2B9750:20FE488BE80074B9]:0)
[junit4] > at java.io.BufferedOutputStream.write(BufferedOutputStream.java:128)
[junit4] > at org.apache.lucene.store.OutputStreamIndexOutput.writeBytes(OutputStreamIndexOutput.java:51)
[junit4] > at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:125)
[junit4] > at org.apache.lucene.codecs.compressing.LZ4.encodeLiterals(LZ4.java:157)
[junit4] > at org.apache.lucene.codecs.compressing.LZ4.encodeLastLiterals(LZ4.java:162)
[junit4] > at org.apache.lucene.codecs.compressing.LZ4.compress(LZ4.java:252)
[junit4] > at org.apache.lucene.codecs.compressing.CompressionMode$LZ4FastCompressor.compress(CompressionMode.java:161)
[junit4] > at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.flush(CompressingStoredFieldsWriter.java:233)
[junit4] > at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.finishDocument(CompressingStoredFieldsWriter.java:166)
[junit4] > at org.apache.lucene.index.DefaultIndexingChain.finishStoredFields(DefaultIndexingChain.java:269)
[junit4] > at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:363)
[junit4] > at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:222)
```
---
Migrated from [LUCENE-5845](https://issues.apache.org/jira/browse/LUCENE-5845) by Robert Muir (@rmuir)
Attachments: [LUCENE-5845_test.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-5845/LUCENE-5845_test.patch)
Contributor guide
Assessment
This issue has not been assessed yet.