Using SSD instead of SATA to store flink state data, application runs slower, and the throughput of read-ops is close to zero per second.
- Dominant language
- C++
- Stars
- 32.1k
- Forks
- 6.9k
- Avg merge
- 32m
- Merged PRs (30d)
- 1
Description
> Note: Please use Issues only for bug reports. For questions, discussions, feature requests, etc. post to dev group: https://groups.google.com/forum/#!forum/rocksdb or https://www.facebook.com/groups/rocksdb.dev
Hi, community!
I use Apache Flink 1.13.5 to handle lots of data, and Flink uses embedded RocksDB to store internal state data. The jar name of embedded RocksDB is frocksdbjni-5.17.2-ververica-2.1.jar, and i don't known this community is responsible or familiar with this domain, and i especially expect that community provide with some advices.
- using SATA
- average read-throughput: 490KB/s, write-throughput: 10.9MB/s
- average read-ops: 0.86/s, write-ops: 57.72/s
- configurations
- write buffer size: 128MB
- max writer buffer number: 8
- block size: 64KB
- block cache size: 256M
- max size level base: 256M
- dynamic level size: true
- min write buffer to merge: 3
- background thread: 8
- max open file: -1
- using SSD
- average read-throughput: 0.35KB/s, write-throughput: 13.57MB/s
- average read-ops: 0/s, write-ops: 45.6/s
- configurations
- increase parallelism: 4
- fsync: false
- info log level: HEAD_LEVEL
- stats dump period sec: 0
- max open file: -1
- even if i add other configurations like writer-buffer-size and writer-buffer-number...., the throughput and iops of SSD don't get any better, the read-ops is still close to zero.
### Expected behavior
I expect to improve the read-writer performance of rocksdb by using SSD instead of SATA, so as to improve the performance of Flink application.
### Actual behavior
Actually, the read-ops and read-throughput of RocksDB based on SSD is more fewer than RocksDB based on SATA, and the throughput of flink application based on SSD is decreased 30~40% than same data and same application based on SATA.
So, do i miss any information? how to take advantage of SSD? Thanks for your any replies or suggestions.
### Steps to reproduce the behavior
Contributor guide
Assessment
This issue has not been assessed yet.