apache / apache/lucene

PatternReplaceCharFilter crashes JVM with OutOfMemoryError [LUCENE-6079]

Open
#7,141 1 comment 0 reactions 0 assignees View on GitHub
affects-version:4.10.2 legacy-jira-priority:Critical module:analysis type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

PatternReplaceCharFilter fills memory with input data until an OutOfMemoryError is thrown.

java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3332)
at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:137)
at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:121)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:569)
at java.lang.StringBuilder.append(StringBuilder.java:190)
at org.apache.lucene.analysis.pattern.PatternReplaceCharFilter.fill(PatternReplaceCharFilter.java:84)
at org.apache.lucene.analysis.pattern.PatternReplaceCharFilter.read(PatternReplaceCharFilter.java:74)
...

PatternReplaceCharFilter should read data chunk-wise and pass the transformed output chunk-wise to the caller.

---
Migrated from [LUCENE-6079](https://issues.apache.org/jira/browse/LUCENE-6079) by Alexander Veit
Environment:
```
Microsoft Windows, x86_64, 32 GB main memory
```

Contributor guide

Open the contributing guide

Research direction

Inspect PatternReplaceCharFilter.java, especially read and fill at the stack-trace lines, to understand how input accumulates. Reproduce the failure with large input and verify that transformed output is delivered chunk-wise without unbounded buffering or an OutOfMemoryError.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.