facebook / facebook/rocksdb

Alternative layouts for SST files

Open
#9,397 1 comment 4 reactions 0 assignees View on GitHub
design discussion feature-request
Dominant language
C++
Stars
32.1k
Forks
6.9k
Avg merge
32m
Merged PRs (30d)
1

Description

@siying shared a post about HTAP. It reminded me of the Ressi data layout in [Spanner](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/46103.pdf), which is based on [PAX](https://www.pdl.cmu.edu/PDL-FTP/Database/pax.pdf).

After moving from local storage to cloud storage, all SST files are stored in the cloud filesystem. Similar to the design of remote compaction contributed by @jay-zhuang, maybe the distributed processing frameworks like Spark can be used to convert the existing SST file formats to some alternative format like PAX for experiments.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.