Fix comment explaining reason for 8KB buffered/chunked writes [LUCENE-9503]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
FSDirectory uses an 8KB buffer and chunk size when writing to disk. The class has a comment explaining how file channels would do mallocs for larger chunks. However, that comment was originally introduced when RandomAccessFile and later FileOutputStream was used. Since the switch to file channels, there is no longer any difference (except the size of the buffer) for different buffer/chunk sizes.
Attached patch keeps the comment, clarifying that it is a historic reason.
Including my research here for background information in case this helps validate the change:
1. The stack buffer for 8KB or below in the JDK is here:
This is used by both `FileOutputStream` and `RandomAccessFile`.
1. This was applicable when the comment to lucene was added and later changed to `FileOutputStream`:
1. Lucene cut over to nio here:
and the comment was later changed to say `file channels` here:
1. File channels use thread local direct buffers instead, see sun.nio.ch.IOUtil.write, which uses Util.getTemporaryDirectBuffer to get a thread local buffer.
---
Migrated from [LUCENE-9503](https://issues.apache.org/jira/browse/LUCENE-9503) by Henning Andersen (@henningandersen)
Attachments: [LUCENE-8kb-comment.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-9503/LUCENE-8kb-comment.patch)
Contributor guide
Research direction
Start in FSDirectory at the comment describing the 8KB buffer and chunk size, then read the attached LUCENE-8kb-comment.patch and the cited history for context. Done means the comment clearly identifies the malloc explanation as historical while accurately reflecting the current file-channel implementation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Documentation
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100