apache / apache/pinot

Streaming query client

Open
#11,901 2 comments 3 reactions 0 assignees View on GitHub
multi-stage pinot-client stale
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 55m
Merged PRs (30d)
182

Description

This pops up as a requirement when quickstart join query might generates many rows. It may blow up broker memory as it tries to keep all the rows in memory then return it back.

The goal of this PR is to provide a grpc based query client. And we can also use it to replace controller console.

This client should config a timeout for broker so it won't hold the resources on server side.

cc: @walterddr

Contributor guide

Open the contributing guide

Research direction

Start by tracing the quickstart join query and the controller console query path to understand how rows are currently returned and where broker resources are held. Then review the proposed gRPC query client and timeout requirements. Done means the scope and interface are defined, large results can be streamed without retaining all rows in broker memory, and broker-side resources are bounded by a timeout.

Written by the indexing model from the issue text.

Assessment

Tech stack
grpc, java
Domain
api, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.