facebook / facebook/rocksdb

Extreme seek performance degradation with deletes

Open
#5,265 13 comments 8 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
32.1k
Forks
6.9k
Avg merge
32m
Merged PRs (30d)
1

Description

Rocksdb iter seek slows down dramatically when there are many large (10MB+) deleted rows. This slow down persists until a compaction occurs. Tested with 5.15.10 and 6.0.1 on an SSD.

### Expected behavior
Seeking to a deleted key should be comparable in performance to seeking to an existing key.

### Actual behavior
This [sample app](https://gist.github.com/theimpostor/12a3f65179ed2e460a171fd33660cb27) demonstrates the behavior. It inserts 286 rows of 16MB each, deletes 256 rows from the middle, and attempts to seek to first key, then first deleted key, and finally the first key after deleted range. Seeking to the first deleted key takes 100x longer than seeking to an existent key:

```
./slow-seek --db db
rocksdb_open [db] took 14 milliseconds
done writing 256 keys of 16777216 size, took 11696 milliseconds
done deleting 256 keys of 16777216 size, took 2 milliseconds
iter seek to key aaaa took 35 milliseconds, return aaaa0000
iter seek to key adel took 1230 milliseconds, return azzz0000
iter seek to key azzz took 18 milliseconds, return azzz0000
rocksdb_close took 2 milliseconds
```

Options file and rocksdb log output are included in the gist.

Later on after the db is compacted the seek times return to normal.

Are there any options I can enable to mitigate this behavior?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.