apache / apache/druid

Standardized way to make intra-cluster (REST) queries

Open
#8,443 1 comment 0 reactions 0 assignees View on GitHub
Area - Dev Contributions Welcome Dev Productivity
Dominant language
Java
Stars
14.1k
Forks
3.8k
Avg merge
2d 58m
Merged PRs (30d)
233

Description

### Problem
While we don't have nice Java API binding, as, for example, gRPC provides, intra-cluster queries are done in various ad-hoc ways, for example, using manual String formatting:

https://github.com/apache/incubator-druid/blob/984958122b962ab019a590f4fd01792de81b75df/server/src/main/java/org/apache/druid/client/DirectDruidClient.java#L150-L151

Or, constructing URLs in static methods:

https://github.com/apache/incubator-druid/blob/984958122b962ab019a590f4fd01792de81b75df/server/src/main/java/org/apache/druid/server/lookup/cache/LookupCoordinatorManager.java#L717-L735

This creates difficulty finding places in the code where a certain Druid API is called. Ideally, the experience should not be significantly worse than IDE's "Find usages" action.

### Proposed solution

Every REST method (`@GET`/`@POST`-annotated) which is called within the cluster has an accompanying static method for forming an URL to call the method from a remote node. This accompanying static method is placed in a centralized place, such as `IntraClusterCalls` class. Javadocs for the REST method and the accompanying static method link to each other for easy navigation. All intra-cluster calls should be made using these static methods for creating URLs.

Contributor guide

Open the contributing guide

Research direction

Start by reading the intra-cluster URL construction in DirectDruidClient.java and LookupCoordinatorManager.java, then compare those callers with the REST methods they invoke. Review the proposed IntraClusterCalls class and the linked REST endpoints. Done means intra-cluster calls use centralized URL-building methods and the REST methods and helpers link to each other in Javadocs.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, backend, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.