apache / apache/lucene

Merging should pass correct fieldinfos to producers always [LUCENE-5990]

Open
#7,052 2 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

I think its been a longstanding issue, but I noticed it in the bulk merge code today and see that we can fix it...

instead of:

```Java
DocumentStoredFieldVisitor visitor = new DocumentStoredFieldVisitor();
storedFieldsReader.visitDocument(docID, visitor);
Document doc = visitor.getDocument();
addDocument(doc, mergeState.mergeFieldInfos);
```

we should do:

```Java
addDocument(doc, mergeState.fieldInfos[i]);
```

This is a lot more consistent and reduce the possibility of scary bugs during merge because the codec does something strange. We should look into all merge logic to see if it can be improved here.

---
Migrated from [LUCENE-5990](https://issues.apache.org/jira/browse/LUCENE-5990) by Robert Muir (@rmuir)

Contributor guide

Open the contributing guide

Research direction

Start with the bulk merge code around DocumentStoredFieldVisitor, storedFieldsReader.visitDocument, addDocument, and mergeState. Inspect the other merge paths for the same fieldInfos handling; done means producers consistently receive the per-segment fieldInfos rather than the shared mergeFieldInfos.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.