google / google/leveldb

Is there a contradiction between group commit and non-sync mode?

Open
#892 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
39.4k
Forks
8.2k
PR merge metrics
No merged PRs in 30d

Description

According to [this commit](https://github.com/google/leveldb/commit/d79762e27369365a7ffe1f2e3a5c64b0632079e1), the introduction of group commit aims to increase the write bandwidth of WAL by I/O in sync mode. However, in the non-sync mode, which is the default option and used more commonly, WAL is usually a pure memory operation without any I/O, similar to `write()` without `fsync()`. In the implementation of DBImpl::Write, the `Writer` object must go through the `BuildBatchGroup()` process, whether it's sync mode or not.
In the non-sync mode, will batching on memory stores introduce more overhead than directly writing WAL to memory in the non-sync mode?

A simple test shows that when the number of threads is small and the value size is small, v1.2 leveldb without group commit performs better than the latest leveldb with group commit.

w/o group commit
```bash
$ ./db_bench --benchmarks=fillseq --num=1000000 --threads=2 --value_size=100
LevelDB: version 1.2
Date: Sat Apr 3 13:05:04 2021
CPU: 8 * Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz
CPUCache: 8192 KB
Keys: 16 bytes each
Values: 100 bytes each (50 bytes after compression)
Entries: 1000000
RawSize: 110.6 MB (estimated)
FileSize: 62.9 MB (estimated)
------------------------------------------------
fillseq : 3.007 micros/op; 72.6 MB/s
```

w/ group commit
```bash
LevelDB: version 1.23
Date: Sat Apr 3 13:05:15 2021
CPU: 8 * Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz
CPUCache: 8192 KB
Keys: 16 bytes each
Values: 100 bytes each (50 bytes after compression)
Entries: 1000000
RawSize: 110.6 MB (estimated)
FileSize: 62.9 MB (estimated)
------------------------------------------------
fillseq : 4.603 micros/op; 48.0 MB/s
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.