apache / apache/lucene

Should NIOFSDir use direct ByteBuffers? [LUCENE-2056]

Open
#3,132 7 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Minor module:core/store type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

I'm trying to test NRT performance, and noticed when I dump the thread stacks that the darned threads often seem to be in `java.nio.Bits.copyToByteArray(Native Method)`... so I wondered whether we could/should use direct ByteBuffers, and whether that would gain performance in general. We currently just use our own byte[] buffer via BufferedIndexInput.

It's hard to test since it's likely platform specific, but if it does result in gains it could be an easy win.

---
Migrated from [LUCENE-2056](https://issues.apache.org/jira/browse/LUCENE-2056) by Michael McCandless (@mikemccand), updated Jun 16 2010
Attachments: [LUCENE-2056.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-2056/LUCENE-2056.patch)

Contributor guide

Open the contributing guide

Research direction

Start by reading the NIOFSDir and BufferedIndexInput implementations referenced in the issue, then review the attached LUCENE-2056.patch. Reproduce the NRT thread-stack observation and compare direct ByteBuffers with the current byte[] buffer across relevant platforms; done means a measured performance conclusion and any required implementation and regression coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.