facebook / facebook/rocksdb

Merge operation performance relative to scan

Open
#12,045 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
32.1k
Forks
6.9k
Avg merge
32m
Merged PRs (30d)
1

Description

I am trying to build an inverted index on top of rocksdb .
There are two implementation that comes to my mind
1) for a given term : store the sorted posting list in a single K-V : key is the term , value is sorted posting list , requires one lookup to get the posting list , the idea is as i stream through data i use merge operation
2) create multiple keys : term#doc_id as the key , value is probably empty. This way i would need to scan a given prefix.

For the first operation , is there some particular merge operator i will need to implement , basically i want a sorted list , or unordered list should also be fine . Are there any existing operators
For the second operation my concern is regarding if the scan operation takes too long.

Are there any other alternatives in general on how to model inverted index on top of rocksdb (i assume this is a common use case)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.