DocValues impls should read all headers up-front instead of per-directsource [LUCENE-4539]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
Currently, when DocValues opens, it just opens files. it doesnt read codec headers etc.
Instead we read these every single time a directsource opens.
I think it should work like PostingsReaders: e.g. the PackedInts impl would read its versioning info and codec headers and creating a new Direct impl should be a IndexInput.clone() + getDirectReaderNoHeader().
Today its much more costly.
---
Migrated from [LUCENE-4539](https://issues.apache.org/jira/browse/LUCENE-4539) by Robert Muir (@rmuir), 1 vote, updated Nov 06 2012
Attachments: [LUCENE-4539.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-4539/LUCENE-4539.patch)
Contributor guide
Research direction
Start by tracing DocValues opening and the DirectSource implementations, then compare their header handling with PostingsReaders. Read the PackedInts versioning path and the uses of IndexInput.clone() and getDirectReaderNoHeader(); done means headers are read once up front while newly created Direct implementations reuse the cloned input without rereading them.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, performance
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100