Tiered storage support
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 2d 55m
- Merged PRs (30d)
- 182
Description
Motivation:
Consider use cases which have Pinot tables that have very large retention, but typically only query the most recent segments. Or use cases where very low latency is expected when querying the recent data, and are more tolerant of slightly higher latencies when querying historical data which is very old. In order to optimize for low latency, we often recommend using high performance SSDs. But if such a use case has 2 years of data, and need the high performance only when querying 1 month of data, it might become desirable to keep only the recent time ranges on SSDs, and keep the less frequently queried ones on cheaper and modest nodes (say HDDs).
Design proposal doc: https://docs.google.com/document/d/1Z4FLg3ezHpqvc6zhy0jR6Wi2OL8wLO_lRC6aLkskFgs/edit#
Contributor guide
Assessment
This issue has not been assessed yet.