facebook / facebook/rocksdb

Creating and dropping column families has gotten much slower

Open
#5,117 11 comments 0 reactions 1 assignee Claimed by @mrambacher View on GitHub
Dominant language
C++
Stars
32.1k
Forks
6.9k
Avg merge
32m
Merged PRs (30d)
1

Description

### Expected behavior
Creating and dropping column families should be fast.

### Actual behavior
It used to be fast in version 4.1-4.9, but has since gotten a lot slower. I've tested this across several versions and it looks like it has gotten increasingly worse.

AFAICT the behaviour is not entirely linear either. Here's some timings that've gotten from 5.17.2:

100 column families: 10.7 seconds
200 column families: 40.5 seconds
400 column families: 2 minutes 42 seconds

### Steps to reproduce the behavior
I've created a simple Maven project that lets one reproduce the problem easily: https://github.com/grove/bugs-and-regressions/tree/master/rocksdb/column-families-create-drop

Clone the repository and run "mvn clean test" in the rocksdb/column-families-create-drop directory. This will run the unit test with RocksDB 5.17.2, and it will take around 40 seconds to complete (creates 200 column families, adds one key/value pair, looks it up again and then drops the column families).

If you change the version number in the pom.xml to be 4.1.0, then the test will complete in around 3 seconds.

Another thing I've observed is that if you create and then drop a column family, i.e and do not create others in the meantime, then it is fast.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.