apache / apache/lucene

Can we use DocValuesSkipper for range facets?

Open
#16,249 0 comments 1 reaction 0 assignees View on GitHub
type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

When I get some spare time, I am planning to take a look into using DocValuesSkipper for range facets (ideally in the sandbox facets if possible).

As reference, it is already being used in [HistogramCollector.java](https://github.com/apache/lucene/blob/main/lucene/sandbox/src/java/org/apache/lucene/sandbox/facet/plain/histograms/HistogramCollector.java).

Some extra background info I lifted about DocValuesSkipper:
```
the histogram collector bulk-counts whole blocks of docs using the doc-values skip index; do the same for range facets. When a dense skip-block's value range falls entirely inside one facet range, we can add the block's doc count in one shot instead of reading doc values and binary-searching per document.
```

Edit 16/6: I think we should start with LongRangeFacetCutter, since it stores raw longs as doc-values so the skipper's min/max will map to the range boundaries (unlike doubles, which has encoded DVs).

Reading HistogramCollector, it uses the DocValuesSkipper in `collect(int doc)` (per-doc) and `collect(DocIdStream stream)` (whole block at once). For range facets, we can do the per-doc part inside advanceExact(int doc), using the skipper to skip the value lookup (and the binary search) when a dense block falls entirely within one interval.

Contributor guide

Open the contributing guide

Research direction

Start with HistogramCollector.java and LongRangeFacetCutter, then inspect the collect(int doc), collect(DocIdStream stream), and advanceExact(int doc) entry points mentioned in the issue. Compare how DocValuesSkipper handles dense blocks and make range facets avoid unnecessary value lookups while preserving correct facet counts.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
performance, search
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.