apache / apache/lucene

Term(Ord)ValComparator.value return BytesRefs that are re-used internally [LUCENE-6808]

Open
#7,866 9 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

While working on [SOLR-6168](https://issues.apache.org/jira/browse/SOLR-6168), which involves some non-trivial usage of `FieldComparator`, I discovered some weird bugs anytime I was using TermOrdValComparator. I ultimately tracked this down to the fact that the `BytesRef` instances returned by `TermOrdValComparator.value(int slot)` are backed by `BytesRefBuilder` instances that the Comparator hangs on to and re-uses – so the values a caller gets back from `FieldComparator.value(slot)` might be changed out from under it before it has a chance to use that value in something like `FieldComparator.compareValues(first,second)`.

The general approach when dealing with BytesRef instances (as i understand it) is that the caller is responsible for making a copy if it wants to hang on to it – but in this case that would violate the generic API of FieldComparator – callers would have to pay attention to when a `FieldComparator` is a
`FieldComparator` and do casting to copy the BytesRef.

It seems like the right solution is for `TermOrdValComparator.value(slot)` (and `TermValComparator.value(slot)` which has a similar BytesRef usage) to return `BytesRef.deepCopyOf(values\[slot\])`

---
Migrated from [LUCENE-6808](https://issues.apache.org/jira/browse/LUCENE-6808) by Chris M. Hostetter (@hossman), updated Sep 22 2015
Attachments: [LUCENE-6808.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-6808/LUCENE-6808.patch) (versions: 2)
Linked issues:
- [SOLR-6168](https://issues.apache.org/jira/browse/SOLR-6168)

Contributor guide

Open the contributing guide

Research direction

Start by locating TermOrdValComparator.value(int slot) and TermValComparator.value(int slot), then inspect how FieldComparator.value(slot) and compareValues(first, second) use the returned BytesRef instances. The issue proposes deep-copying the slot value; done means returned values remain stable when the comparator reuses its internal BytesRefBuilder-backed values, with relevant comparator behavior covered by tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.