Improve Norms merging APIs [LUCENE-3787]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
Spinoff from #3706, since it took us a fair amount of time to track down,
I think its worth trying to improve the API.
The DocValuesConsumer api's default merge implementation calls canMerge/getDocValuesForMerge/getDocValuesType (protected methods).
but its a little strange how this works:
- all norms implementations must override the default DV implementation, or they might accidentally merge docvalues into their norms!
- preflex-RW only overrides 2 of these... how is its norms merging working... is it?
- its tricky obviously for issues like #3706: as delegating merge() is not very obvious either.
So I think we should look at this, instead of having all NormsWriters override DocValues, redefining these methods in a way thats
not a "is-a" relationship, we could do something else like have split NormsConsumer/DVConsumer apis that share a package-private base class.
This way a norms impl just extends NormsConsumer and there are no traps.
---
Migrated from [LUCENE-3787](https://issues.apache.org/jira/browse/LUCENE-3787) by Robert Muir (@rmuir)
Contributor guide
Research direction
Start by tracing the DocValuesConsumer default merge implementation and its protected canMerge, getDocValuesForMerge, and getDocValuesType methods, then compare how norms implementations and preflex-RW override them. The issue's proposed direction is to separate NormsConsumer and DVConsumer APIs; done should remove the accidental docvalues-merging trap while preserving correct norms merging.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100