TestFieldsReader fails in 4.x with OOM [LUCENE-5066]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
Its FaultyIndexInput is broken (doesn't implement seek/clone correctly).
This causes it to read bogus data and try to allocate an enormous byte[] for a term.
The bug was previously hidden:
FaultyDirectory doesnt override openSlice, so CFS must not be used at flush if you want to trigger the bug.
FailtyIndexInput's clone is broken, it uses "new" but doesn't seek the clone to the right place. This causes a disaster with BufferedIndexInput (which it extends), because BufferedIndexInput (not just the delegate) must "know" its position since it has seek-within-block etc code...
It seems with this test (very simple one), that only 3.x codec triggers it because its term dict relies upon clone()'s being seek'd to right place.
I'm not sure what other codecs rely upon this, but imo we should also add a low-level test for directories that does something like this to ensure its really tested:
```Java
dir.createOutput(x);
dir.openInput(x);
input.seek(somewhere);
clone = input.clone();
assertEquals(somewhere, clone.getFilePointer());
```
---
Migrated from [LUCENE-5066](https://issues.apache.org/jira/browse/LUCENE-5066) by Robert Muir (@rmuir)
Attachments: [LUCENE-5066.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-5066/LUCENE-5066.patch)
Contributor guide
Research direction
Start by reading FaultyIndexInput and FaultyDirectory, then compare their clone, seek, and openSlice behavior with BufferedIndexInput. Reproduce the simple codec test that triggers the bogus term data and OOM. Done means cloned inputs preserve the file pointer and a low-level directory test covers this behavior without the allocation failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100