apache / apache/lucene

Do we suboptimally call ByteBuffersDataOutput.toDataInput()?

Open
#12,852 3 comments 0 reactions 0 assignees View on GitHub
type:task
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

See this comment: https://github.com/apache/lucene/pull/12624#issuecomment-1829633268

> I wonder: are there other places in Lucene that might fall prey to this performance trap (calling toDataInput frequently while continuing to append bytes to the DataOutput)? FreqProxTermsWriter seems to use this for each term to re-read the postings (maybe for the static sort use case).

Contributor guide

Open the contributing guide

Research direction

Start with the comment on pull request 12624, then inspect FreqProxTermsWriter and its use of ByteBuffersDataOutput.toDataInput() for each term. Check whether input recreation occurs while bytes are still being appended; done means identifying any confirmed performance trap and the affected call sites.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
performance, search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.