facebook / facebook/rocksdb

Using SSD instead of SATA to store flink state data, application runs slower, and the throughput of read-ops is close to zero per second.

Open
#10,426 1 comment 0 reactions 0 assignees View on GitHub
question up-for-grabs
Dominant language
C++
Stars
32.1k
Forks
6.9k
Avg merge
32m
Merged PRs (30d)
1

Description

> Note: Please use Issues only for bug reports. For questions, discussions, feature requests, etc. post to dev group: https://groups.google.com/forum/#!forum/rocksdb or https://www.facebook.com/groups/rocksdb.dev

Hi, community!
I use Apache Flink 1.13.5 to handle lots of data, and Flink uses embedded RocksDB to store internal state data. The jar name of embedded RocksDB is frocksdbjni-5.17.2-ververica-2.1.jar, and i don't known this community is responsible or familiar with this domain, and i especially expect that community provide with some advices.

- using SATA
- average read-throughput: 490KB/s, write-throughput: 10.9MB/s
- average read-ops: 0.86/s, write-ops: 57.72/s
- configurations
- write buffer size: 128MB
- max writer buffer number: 8
- block size: 64KB
- block cache size: 256M
- max size level base: 256M
- dynamic level size: true
- min write buffer to merge: 3
- background thread: 8
- max open file: -1
- using SSD
- average read-throughput: 0.35KB/s, write-throughput: 13.57MB/s
- average read-ops: 0/s, write-ops: 45.6/s
- configurations
- increase parallelism: 4
- fsync: false
- info log level: HEAD_LEVEL
- stats dump period sec: 0
- max open file: -1
- even if i add other configurations like writer-buffer-size and writer-buffer-number...., the throughput and iops of SSD don't get any better, the read-ops is still close to zero.

### Expected behavior
I expect to improve the read-writer performance of rocksdb by using SSD instead of SATA, so as to improve the performance of Flink application.

### Actual behavior
Actually, the read-ops and read-throughput of RocksDB based on SSD is more fewer than RocksDB based on SATA, and the throughput of flink application based on SSD is decreased 30~40% than same data and same application based on SATA.

So, do i miss any information? how to take advantage of SSD? Thanks for your any replies or suggestions.

### Steps to reproduce the behavior

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.