Merge two rocksdb instances
- Dominant language
- C++
- Stars
- 32.1k
- Forks
- 6.9k
- Avg merge
- 32m
- Merged PRs (30d)
- 1
Description
We want to implement a merge operation of two rocksdb instance in our distributed table store like hbase.
Firstly, we try repairdb, but we found that repairdb need same cf name with same cf id. It's not a strongly guarantee in rocksdb implements.
Then, we try to use ingest file, but ingest file need has table properties ingest-file.version and ingest-file.global_seq which not exists in normal sst.
So we add ingest-file.version and ingest-file.global_seq in all normal files with version=2 and global_seq=-1 (let rocksdb read key's real seq. If we don't set it, it will read largest seq in manifest. If we set it to other value, it will let it to be global seq, not key's real seq).
But it must need a pwrite which distributed file system often not implements.
It also let rocksdb sst which be checkpointed (hard link) changed.
I think add a flag link ingest-file.version in manifest maybe make sense.
Contributor guide
Assessment
This issue has not been assessed yet.