apache / apache/pinot

Tiered storage support

Open
#5,553 1 comment 2 reactions 0 assignees View on GitHub
PEP-Request
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 55m
Merged PRs (30d)
182

Description

Motivation:
Consider use cases which have Pinot tables that have very large retention, but typically only query the most recent segments. Or use cases where very low latency is expected when querying the recent data, and are more tolerant of slightly higher latencies when querying historical data which is very old. In order to optimize for low latency, we often recommend using high performance SSDs. But if such a use case has 2 years of data, and need the high performance only when querying 1 month of data, it might become desirable to keep only the recent time ranges on SSDs, and keep the less frequently queried ones on cheaper and modest nodes (say HDDs).

Design proposal doc: https://docs.google.com/document/d/1Z4FLg3ezHpqvc6zhy0jR6Wi2OL8wLO_lRC6aLkskFgs/edit#

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.