apache / apache/lucene

Regexp query: escape sequences are treated as character classes [LUCENE-10642]

Open
#11,678 6 comments 0 reactions 0 assignees View on GitHub
affects-version:9.0 affects-version:9.1 affects-version:9.2 affects-version:9.3 legacy-jira-priority:Major type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

Interesting issue has been reported to Opensearch project [1], which has been caused by [2], [3]. In the nutshell, the regression is causing escape sequences (like \n, \r, \t, ...) to be treated as character classes (specifically,

The problematic function is RegExp::matchPredefinedCharacterClass which does not consider characters that denote an escaped construct. Simple test to reproduce which fails with IllegalArgumentException("invalid character class"):

 
```
public class TestRegexpQuery extends LuceneTestCase {
  public void testEscapeSequences() throws IOException {          
assertEquals(1, regexQueryNrHits("\\n"));          
assertEquals(1, regexQueryNrHits("[\\n]"));   }
  }
}
 
```
 

[1]
[2]
[3]

---
Migrated from [LUCENE-10642](https://issues.apache.org/jira/browse/LUCENE-10642) by Andriy Redko (@reta)

Contributor guide

Open the contributing guide

Research direction

Start with RegExp::matchPredefinedCharacterClass and reproduce the failure using the TestRegexpQuery example in the issue. Check the referenced Java regular-expression documentation and commits to understand escaped constructs. Done means queries for "\\n" and "[\\n]" each return one hit without IllegalArgumentException.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.