Alternative layouts for SST files
Open
design discussion
feature-request
- Dominant language
- C++
- Stars
- 32.1k
- Forks
- 6.9k
- Avg merge
- 32m
- Merged PRs (30d)
- 1
Description
@siying shared a post about HTAP. It reminded me of the Ressi data layout in [Spanner](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/46103.pdf), which is based on [PAX](https://www.pdl.cmu.edu/PDL-FTP/Database/pax.pdf).
After moving from local storage to cloud storage, all SST files are stored in the cloud filesystem. Similar to the design of remote compaction contributed by @jay-zhuang, maybe the distributed processing frameworks like Spark can be used to convert the existing SST file formats to some alternative format like PAX for experiments.
Contributor guide
Assessment
This issue has not been assessed yet.