apache / apache/kvrocks

Storage: Investigate O_DIRECT on RocksDB

Open
#3,058 0 comments 2 reactions 0 assignees View on GitHub
enhancement
Dominant language
C++
Stars
4.4k
Forks
658
Avg merge
1d 20h
Merged PRs (30d)
10

Description

### Search before asking

- [x] I had searched in the [issues](https://github.com/apache/kvrocks/issues) and found no similar issues.

### Motivation

Currently, the hardware is becoming faster and faster. `O_DIRECT` is supported by rocksdb [1], and allowed to be enabled by MyRocks and other applications.

Now, for reader, there're two ways to bypass kernel buffer and user buffer:
1. `mmap`, which shares the buffer among kernel and user space
2. O_DIRECT, forcing aligned read

RocksDB has limited support for `mmap`, see [2]. O_DIRECT might enhance performance, but it also has some counter parts, see [3]

[1] https://github.com/facebook/rocksdb/wiki/Direct-IO
[2] https://smalldatum.blogspot.com/2022/05/using-mmap-with-rocksdb.html
[3] https://github.com/facebook/rocksdb/issues/6448

### Solution

The code would be easy, perhaps the most important thing is setting up benchmark on different machines

### Are you willing to submit a PR?

- [x] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the RocksDB Direct-IO documentation and issue 6448, then review the storage read paths relevant to mmap and O_DIRECT. Set up benchmarks on different machines and compare performance, alignment requirements, and trade-offs. Done means documented benchmark results and a clear recommendation about enabling O_DIRECT.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
database, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.