facebook / facebook/rocksdb

Less Performance while Block Cache is ON for higher value sizes

Open
#3,439 3 comments 0 reactions 1 assignee Claimed by @anand1976 View on GitHub
Dominant language
C++
Stars
32.1k
Forks
6.9k
Avg merge
32m
Merged PRs (30d)
1

Description

About Service:
A little bit about our service is we are using RocksDB as OnBox cache in WINDOWS env. (we pinvoke native Rocks C Apis) which is a wrapper around Rocks which limits the data size inside Rocks with LRU based eviction, Expiration capability etc. Since it has some different behavior around how it performs, we have written Stress tool which basically fires read and write calls with certain value sizes of records at certain Transactions Per Second.

Experiment Setup:
The whole setup is working as expected i.e When block cache is enabled it is giving higher TPS than block cache off for less value sizes i.e 1KB to 6KB values. For 20 KB value size, for 90% Reads and 10% Writes scenerio, when block cache is on, the TPS is dropping to around 1000 Transactions Per Second where I can see block cache hits in DB stats and high rocksdb.read.block.get.micros which has avg of 13280.526676 Us, The same when block cache is off, is giving easy 5000 transactions per second (as long as it doesn't go beyond L2) where I can see all the Gets are a DB hit and a very high Disk IO of 300 MBPS. Is this due to mutex locks on LRU Cache shards?. Can someone let me know whats going on or any of my options wrong?

DB stats for with and without Block Cache for first 20 mins of running: https://www.diffchecker.com/YndiLi7s

Rocks Options: (Both have same options except 15 GB allocation for With Block Cache) https://docs.google.com/document/d/1Kc11mDHy3-bO9xLY3eZRASEjxTwoh9NYejg2Nz4d7Hk/edit?usp=sharing
https://www.diffchecker.com/MfWrpFqL

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.