apache / apache/lucene

FVH BaseFragmentsBuilder does not properly support colored pre/post tags

Open
#13,933 0 comments 0 reactions 0 assignees View on GitHub
type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

Given the `BaseFragmentsBuilder` description:

```java
...
/**
* Base FragmentsBuilder implementation that supports colored pre/post tags and multivalued fields.
*
*

Uses {@link BoundaryScanner} to determine fragments.
*/
public abstract class BaseFragmentsBuilder implements FragmentsBuilder {
...
```

We assume that if we input a query and an array of pre and post tags, they will follow the same order, like:

| *Query* | *Pre tag* | *Post tag* |
|---|---|---|
| A B | `` | `` |
| C B | `` | `` |
| C A | `` | `` |

It will not tag in a ordered way as the current `BaseFragmentsBuilder` implementation gets tags in a almost random order:

```java
protected String getPreTag(String[] preTags, int num) {
int n = num % preTags.length;
return preTags[n];
}
```

This is links back to [this issue](https://issues.apache.org/jira/browse/LUCENE-2524).

I already done some initial work to solve a problem where I work, but I would like to have a proper solution for Lucene.

The root cause is in the `FieldQuery` `flatten`, `saveTerms` and `expand` methods. They do need to exist but they also mess the order of pre/post tags. The `termOrPhraseNumber` is used to get the preTag, and should follow the order of the queries.

I will try to add a unit test that properly illustrates this problem as it is kinda complex.

### Version and environment details

Lucene 3.0+

Any environment

Contributor guide

Open the contributing guide

Research direction

Start by examining BaseFragmentsBuilder and the FieldQuery flatten, saveTerms, and expand methods named in the issue. Trace how termOrPhraseNumber is assigned and used for pre/post tags, then add the proposed unit test demonstrating the ordering problem. Done means the test passes with tags following query order.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.