apache / apache/druid

Use smile format instead of json for internal Druid process communication.

Open
#7,798 4 comments 0 reactions 0 assignees View on GitHub
Performance
Dominant language
Java
Stars
14.1k
Forks
3.8k
Avg merge
2d 58m
Merged PRs (30d)
233

Description

For internal communication between Druid processes, switching to a binary format would provide reduction in bytes transferred over the network and better performance.
This should be done for transferring published segments data between coordinator and broker and possibly at other places.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the internal communication that transfers published segments data between the coordinator and broker, then identify the JSON serialization points involved. Check whether other Druid process-to-process communication paths are in scope. Done means the agreed communication paths use Smile and retain the required data exchange behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.