TTL-based Interval Result Level Caching For Time-Ordered Queries Touching Realtime Segments
- Dominant language
- Java
- Stars
- 14.1k
- Forks
- 3.8k
- Avg merge
- 2d 58m
- Merged PRs (30d)
- 233
Description
### Description
Want to support result-set caching of queries hitting realtime data nodes. Want to create a way to partition the result set of a query (from either realtime/historical data nodes) into cacheable granular intervals that can either pulled from cache and stitched into the query result, or issued as a query to data nodes.
Providing a TTL query context header would dictate how "recent" of an interval we'd want to serve from cache, and otherwise query from data nodes. Something like `cacheTTL: "PT1M"` would tell the brokers to serve from cache all results that were > PT1M ago, and issue queries to data nodes for data
Contributor guide
Research direction
No files, tests, or entry points are named. Start by tracing broker query handling and result caching for realtime and historical data nodes, then define how a TTL query context header partitions and stitches intervals. Done means cacheable older intervals are served from cache while intervals newer than the requested TTL are queried from data nodes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend-api-design, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100