Brokers to catch repetitive queries
- Dominant language
- Java
- Stars
- 14.1k
- Forks
- 3.8k
- Avg merge
- 2d 58m
- Merged PRs (30d)
- 233
Description
### Description
Broker registers queries being executed and associates them with `Future` objects. When it notices that a query has arrived that is already being executed, it just puts the HTTP thread to await to the same (already existing) `Future`, instead of sparking a duplicate execution.
This can be seen as a form of results "pre-cache".
### Motivation
When a query executes for a long time, users get impatient and reload a web interface, initiating the same queries to Druid. Currently, by doing so they only make things worse, because brokers (and historicals), still crunching results for the previous query (if no form of [cancellation protocol](https://github.com/apache/incubator-druid/issues/7087) is used), start another instance of the query.
Contributor guide
Research direction
Start by locating the Broker code that registers executing queries and associates them with Future objects. Determine how an arriving duplicate query could await the existing Future, then define and test the behavior for repeated long-running queries without triggering duplicate execution.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend-api-design, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100