MMapDirectory.PRELOAD_HINT should also consider FileTypeHint.INDEX
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
### Description
Currently, `MMapDirectory.PRELOAD_HINT` only looks at the `PreloadHint.INSTANCE`. In my opinion, it should look at `FileTypeHint.INDEX` as well: index files are very small by contract, and file formats expect them to reside in memory, so it makes sense to make them preloaded to avoid cold starts.
An alternative would be to pass the `PreloadHint.INSTANCE` hint to every index file, but I like this alternative less. My mental model is that index files are expected to reside in memory while data files are generally not, and the `PreloadHint.INSTANCE` hint is a flag that should be set on the small minority of data files that need to reside in memory for decent performance.
By the way, maybe `PreloadHint` should be renamed to `InMemoryHint` or something along these lines, to signal that the file is expected to reside in memory rather than that it should be preloaded. And then preloading is a way to help the OS load it in memory at open time instead of waiting for it to naturally get loaded in memory over time as a result of the OS figuring out that all pages of this file are hot.
Contributor guide
Research direction
Start by tracing MMapDirectory.PRELOAD_HINT and the handling of PreloadHint.INSTANCE and FileTypeHint.INDEX. Confirm the existing preload behavior for index and data files, then update the relevant behavior so index files are considered; done means index files are preloaded as described without requiring PreloadHint.INSTANCE on each one.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100