google / google/leveldb

CPU consumption keeps growing while continuously adding key/value with big data.

Open
#849 1 comment 2 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
39.4k
Forks
8.2k
PR merge metrics
No merged PRs in 30d

Description

Test environment is Raspi4 board.

Disk performance is
```
dd if=/dev/zero of=/tmp/output conv=fdatasync bs=2M count=1k; rm -f /tmp/output
1024+0 records in
1024+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 132.399 s, 16.2 MB/s
```
I execute `db_bench` to continuously write about 1M data
```
$ ./db_bench --benchmarks=fillseq --value_size=1048576
LevelDB: version 1.22
Date: Fri Nov 27 15:44:29 2020
CPU: 0 *
CPUCache:
Keys: 16 bytes each
Values: 1048576 bytes each (524288 bytes after compression)
Entries: 1000000
RawSize: 1000015.3 MB (estimated)
FileSize: 500015.3 MB (estimated)
WARNING: Snappy compression is not enabled
...
```
Get CPU usage On another terminal
```
$ ps -p PID -o pcpu=
```
At first, the CPU usage is 19.2%.
After 10 minutes, it grows to 19.7%.
After 10 minutes, it grows to 21.4%
After 20 minutes, it grows to 24.4%

If use smaller data (such as 100KB), CPU usage isn't increased.
Maybe writing speed exceeds disk capacity.

Who know the reason ?
Are there any options of leveldb which can avoid this ?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.