facebook / facebook/rocksdb

CPU overhead on file ingestion

Open
#5,010 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
32.1k
Forks
6.9k
Avg merge
32m
Merged PRs (30d)
1

Description

> Note: Please use Issues only for bug reports. For questions, discussions, feature requests, etc. post to dev group: https://www.facebook.com/groups/rocksdb.dev

### Expected behavior
We have a data import tool which can huge amount of SST files into a DB, one file at a time. When we have many small SST files we see high CPU overhead on ingestion. Profiling shows the overhead comes from reconstruction of `Version`, especially when it memcpy all smallest and largest key to the version's own arena:
![ingest](https://user-images.githubusercontent.com/2606959/53275705-f7243500-36b0-11e9-9200-3b09d0c7387d.png). The arena is introduced in this commit apparently aiming to localize memory access when traversing the LSM tree on Get: https://github.com/facebook/rocksdb/commit/f697cad15 I'm not able to stable reproduce the latency gain mentioned in the commit using readrandom and readwhilewriting, but the optimization does sound reasonable. While we are not able to improve our tool to batch ingest multiple SST file, are there any suggestions that can reduce SST file overhead? Thanks.

### Actual behavior
File ingest cpu overhead.

### Steps to reproduce the behavior
I have a tool to generate 10k SST files and try import them to an empty DB:
master: 16.11s user 4.45s system 77% cpu 26.400 total
with a patch to remove the memcpy: 10.16s user 3.74s system 75% cpu 18.336 total
The tool: https://gist.github.com/yiwu-arbug/cf646e8ef1763455e9132ada98125e32
The patch to remove memcpy: https://gist.github.com/yiwu-arbug/fd7771144972fae1b42466936699d91a

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.