Write Amplification factor for small key-values with frequent writes to levelDB
- Dominant language
- C++
- Stars
- 39.4k
- Forks
- 8.2k
- PR merge metrics
- No merged PRs in 30d
Description
Hi Team,
I am facing a serious issue with write amplification. Consider, There are key-values that will write to levelDB for every 5 seconds. Those key-values are always batched into a write_batch object and then written to levelDB using leveldb_write() for every 5 sec. You can assume total size of key-values will be 500 bytes.
With leveldb ( sync = false ) , I am getting eMMC write count of 221 writes. where, If I write values directly into eMMC by maintaining some offsets for each value, I am getting eMMC write count of 70 writes.
The difference is Huge. Is there any way to optimize leveldb eMMC write count ? FYI, I tried all options available in Options.h file.
Contributor guide
Research direction
Start by reviewing Options.h and the reported leveldb_write() and write_batch usage, then reproduce the five-second, 500-byte write workload with sync disabled. Compare LevelDB's eMMC write count with direct writes and determine whether a specific configuration or documented limitation explains the difference; done requires a validated optimization or a clearly supported conclusion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- databases, performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100