apache / apache/lucene

FunctionQuery ValueSource value computed twice per document [LUCENE-4574]

Open
#5,640 25 comments 0 reactions 1 assignee Claimed by @dsmiley View on GitHub
affects-version:4.0 affects-version:4.1 legacy-jira-priority:Major module:core/search type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

I was working on a custom ValueSource and did some basic profiling and debugging to see if it was being used optimally. To my surprise, the value was being fetched twice per document in a row. This computation isn't exactly cheap to calculate so this is a big problem. I was able to work-around this problem trivially on my end by caching the last value with corresponding docid in my FunctionValues implementation.

Here is an excerpt of the code path to the first execution:

```
at org.apache.lucene.queries.function.docvalues.DoubleDocValues.floatVal(DoubleDocValues.java:48)
at org.apache.lucene.queries.function.FunctionQuery$AllScorer.score(FunctionQuery.java:153)
at org.apache.lucene.search.TopFieldCollector$OneComparatorScoringMaxScoreCollector.collect(TopFieldCollector.java:291)
at org.apache.lucene.search.Scorer.score(Scorer.java:62)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:588)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:280)
```

And here is the 2nd call:

```
at org.apache.lucene.queries.function.docvalues.DoubleDocValues.floatVal(DoubleDocValues.java:48)
at org.apache.lucene.queries.function.FunctionQuery$AllScorer.score(FunctionQuery.java:153)
at org.apache.lucene.search.ScoreCachingWrappingScorer.score(ScoreCachingWrappingScorer.java:56)
at org.apache.lucene.search.FieldComparator$RelevanceComparator.copy(FieldComparator.java:951)
at org.apache.lucene.search.TopFieldCollector$OneComparatorScoringMaxScoreCollector.collect(TopFieldCollector.java:312)
at org.apache.lucene.search.Scorer.score(Scorer.java:62)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:588)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:280)
```

The 2nd call appears to use some score caching mechanism, which is all well and good, but that same mechanism wasn't used in the first call so there's no cached value to retrieve.

---
Migrated from [LUCENE-4574](https://issues.apache.org/jira/browse/LUCENE-4574) by David Smiley (@dsmiley), updated Jun 08 2022
Attachments: [LUCENE-4574.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-4574/LUCENE-4574.patch) (versions: 4), [Test_for_LUCENE-4574.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-4574/Test_for_LUCENE-4574.patch)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.