apache / apache/lucene

Query parser fails to parse a range query string when there are escaped brackets inside the range

Open
#13,234 2 comments 0 reactions 0 assignees View on GitHub
type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

Assume there's a query parser created, e.g.:

```java
Analyzer analyzer = new ClassicAnalyzer();
QueryParser queryParser = new QueryParser( "field", analyzer );
```

trying to parse simple ranges like:

```java
// simple range query with no escapes works fine:
Query query = queryParser.parse( "[ 1 TO 10 ]" );
```
works as expected and a `TermRangeQuery` is created with no exceptions.

But if the range in the string contains some escaped brackets -- it leads to a parsing exception.
Let's assume one would want to extend the query parser to work with date-time fields and would want to parse something like:

```java
// another range query but now it has some escaping between the range brackets:
query = queryParser.parse( "[ 2024\\-01\\-01T01\\:01\\:01\\+01\\:00\\[Europe\\/Warsaw\\] TO 2025\\-01\\-01T01\\:01\\:01\\+01\\:00\\[Europe\\/Warsaw\\] ]" );
```

where all special characters are escaped, leads to:
```java
Caused by: org.apache.lucene.queryparser.classic.ParseException: Encountered " "]" "] "" at line 1, column 50.
Was expecting:
"TO" ...

at org.apache.lucene.queryparser.classic.QueryParser.generateParseException(QueryParser.java:1004)
at org.apache.lucene.queryparser.classic.QueryParser.jj_consume_token(QueryParser.java:867)
at org.apache.lucene.queryparser.classic.QueryParser.Term(QueryParser.java:532)
at org.apache.lucene.queryparser.classic.QueryParser.Clause(QueryParser.java:366)
at org.apache.lucene.queryparser.classic.QueryParser.Query(QueryParser.java:251)
at org.apache.lucene.queryparser.classic.QueryParser.TopLevelQuery(QueryParser.java:223)
at org.apache.lucene.queryparser.classic.QueryParserBase.parse(QueryParserBase.java:137)
... 4 more
```

Note, the idea to do range queries for dates is to have something along the lines:
```java
QueryParser queryParser = new QueryParser( "field", analyzer ) {
@Override
protected Query newRangeQuery(String field, String part1, String part2, boolean startInclusive, boolean endInclusive) {
var p1 = parseValue(part1);
var p2 = parseValue(part2);

return createRangeQueryForDates( field, p1, p2, startInclusive, endInclusive );
}
};
```
but because of the parsing error described above, execution never reaches this point.

### Version and environment details

Java version: 17.0.9, vendor: Amazon.com Inc.
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "6.7.10-200.fc39.x86_64", arch: "amd64", family: "unix"

Lucene 9.10.0

Contributor guide

Open the contributing guide

Research direction

Reproduce the escaped-bracket range with QueryParser.parse and trace the Term parsing path identified in QueryParser.java, especially the failure around the closing bracket. Compare it with the simple range example and verify that the date-style input is accepted as one range and reaches the overridden newRangeQuery method without a ParseException.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.