apache / apache/lucene

Prefix and postfix all extracted phrases when phrase is not complete [LUCENE-4441]

Open
#5,507 0 comments 0 reactions 0 assignees View on GitHub
affects-version:3.6.1 affects-version:4.0-BETA legacy-jira-label:highlight legacy-jira-priority:Trivial module:highlighter type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

Heya,
I would like to be able to prefix and postfix phrases when extracted text is not a complete phrase.

For example, let's say I have a phrase like

```none

"Here is my text"
```

and another like

```none

"This is a very long text and I want that Lucene highlight it even if this content is very long."
```

When highlighting "text", I get something like:

```html

"Here is my text"
"This is a very long text and I want that Lucene highlight"
```

What I would like to get back is:

```html

"Here is my text"
"This is a very long text and I want that Lucene highlight [...]"
```

As you can see, the " [...]" is added at the end because we extracted content from the field and not all the field.

So, we can add some options to Highlighter (FastVectorHighlighter) as follow:

```java
// Default values :
// preLine = null;
// postLine = null;
public final String[] getBestFragments( final FieldQuery fieldQuery, IndexReader reader, int docId,
String fieldName, int fragCharSize, int maxNumFragments,
FragListBuilder fragListBuilder, FragmentsBuilder fragmentsBuilder,
String[] preTags, String[] postTags, String preLine, String postLine, Encoder encoder ) throws IOException;
```

I hope this is doable. :-/

---
Migrated from [LUCENE-4441](https://issues.apache.org/jira/browse/LUCENE-4441) by David Pilato (@dadoonet)

Contributor guide

Open the contributing guide

Research direction

The issue names FastVectorHighlighter and its getBestFragments overload; start there and trace how extracted fragments determine whether the full field was returned. Done means the requested preLine/postLine markers appear for incomplete extracted phrases while complete-phrase behavior remains unchanged; the payload names no test file, so locate the relevant highlighter tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, search
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.