Custom attributes get cleared by SynonymFilter [LUCENE-6667]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
I believe the Lucene API enables users to define their custom attributes (by extending `AttributeImpl`) which may be added by custom Tokenizers.
It seems, the `clear` and `copyTo` methods must be implemented to clear and restore the state of this custom attribute.
However, some filters (in our case the SynonymFilter) simply call `AttributeSource.clearAttributes` without invoking `copyTo`. Instead the filter just resets some known attributes, simply ignoring all other custom attributes. In the end our custom attribute value is lost.
Is this a bug in `SynonymFilter` (and others) or are we using the API in the wrong way?
A solution might be of course to provide empty implementations of `clear` and `copyTo`, but I'm not sure if this has other unwanted effects.
---
Migrated from [LUCENE-6667](https://issues.apache.org/jira/browse/LUCENE-6667) by Oliver Becker
Contributor guide
Research direction
Start by tracing SynonymFilter's use of AttributeSource.clearAttributes and compare it with the custom AttributeImpl clear and copyTo contract described in the issue. Reproduce the loss of a custom attribute through the filter, then confirm the API behavior and add coverage if the intended behavior is established.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100