apache / apache/druid

TTL-based Interval Result Level Caching For Time-Ordered Queries Touching Realtime Segments

Open
#18,602 2 comments 0 reactions 0 assignees View on GitHub
Area - Cache Area - Streaming Ingestion Feature/Change Description Performance
Dominant language
Java
Stars
14.1k
Forks
3.8k
Avg merge
2d 58m
Merged PRs (30d)
233

Description

### Description

Want to support result-set caching of queries hitting realtime data nodes. Want to create a way to partition the result set of a query (from either realtime/historical data nodes) into cacheable granular intervals that can either pulled from cache and stitched into the query result, or issued as a query to data nodes.

Providing a TTL query context header would dictate how "recent" of an interval we'd want to serve from cache, and otherwise query from data nodes. Something like `cacheTTL: "PT1M"` would tell the brokers to serve from cache all results that were > PT1M ago, and issue queries to data nodes for data

Contributor guide

Open the contributing guide

Research direction

No files, tests, or entry points are named. Start by tracing broker query handling and result caching for realtime and historical data nodes, then define how a TTL query context header partitions and stitches intervals. Done means cacheable older intervals are served from cache while intervals newer than the requested TTL are queried from data nodes.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend-api-design, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.