apache / apache/lucene

It seems that escaped query characters are not treated as escaped when calling queryParser.Parse()

Open
#13,962 0 comments 0 reactions 0 assignees View on GitHub
type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

Having following code:
```java
package org.example;

import org.apache.lucene.analysis.Analyzer;
import org.apache.lucene.analysis.TokenStream;
import org.apache.lucene.analysis.Tokenizer;
import org.apache.lucene.analysis.core.LowerCaseFilter;
import org.apache.lucene.analysis.miscellaneous.ASCIIFoldingFilter;
import org.apache.lucene.analysis.standard.StandardFilter;
import org.apache.lucene.analysis.standard.StandardTokenizer;
import org.apache.lucene.util.Version;

import java.io.Reader;

public class DefaultQA extends Analyzer
{
private Version _version = Version.LUCENE_48;

@Override
protected TokenStreamComponents createComponents(String s, Reader reader) {
Tokenizer tokenizer = new StandardTokenizer(_version, reader);
TokenStream result = new StandardFilter(_version, tokenizer);
result = new LowerCaseFilter(_version, result);
result = new ASCIIFoldingFilter(result);

return new TokenStreamComponents(tokenizer, result);
}
}
```

```java
package org.example;

import org.apache.lucene.queryparser.classic.ParseException;
import org.apache.lucene.queryparser.classic.QueryParser;
import org.apache.lucene.util.Version;

public class Main {
public static void main(String[] args) throws ParseException {
var qa = new DefaultQA();

var queryParser = new QueryParser(Version.LUCENE_48, "fieldName", qa);
queryParser.setDefaultOperator(QueryParser.Operator.AND);

var query = QueryParser.escape("more&more"); //this is the query I am trying to "parse"

var searchQuery = queryParser.parse(query);
var searchQuery2 = queryParser.createBooleanQuery("fieldName", query);
var searchQuery3 = queryParser.createPhraseQuery("fieldName", query);

System.out.println(searchQuery);
System.out.println(searchQuery2);
System.out.println(searchQuery3);
}
}
```

I am getting this result:
```
+fieldName:more +fieldName:more
fieldName:more fieldName:more
fieldName:"more more"
```

But what I expected was escaped query:
```
+fieldName:more\&more
fieldName:more\&more
fieldName:"more\&more"
```

It seems as bug to me that escaped query doesn't stay escaped.

Original issue was found on Lucenenet (Lucene.NET) port ([https://github.com/apache/lucenenet/issues/850](https://github.com/apache/lucenenet/issues/850))

### Version and environment details

`Java Lucene 4.8`, but it was allegedly tested even in `Lucene 10.0.0` with the same results

Contributor guide

Open the contributing guide

Research direction

The report names QueryParser.escape(), QueryParser.parse(), createBooleanQuery(), and createPhraseQuery(), but no repository files or tests. Start by tracing those entry points and the analyzer behavior for escaped ampersands; done should be a clearly decided and tested treatment of escaped query characters across the reported methods.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.