apache / apache/pinot

Web Socket Endpoints for querying in Pinot Broker

Open
#11,024 3 comments 1 reaction 0 assignees View on GitHub
feature
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 55m
Merged PRs (30d)
182

Description

Currently, we have REST API endpoints in pinot broker for querying. However, adding a WebSocket endpoint helps with having low latency & high throughput queries once the connection is established between the pinot broker and the client service.

**Use Case:**
Using WebSocket to connect to a Pinot table that is being populated from a Kafka topic / some streaming source. By not directly connecting to the Kafka topic, we can take advantage of data manipulation and efficient filtering, and aggregation features in Pinot. This approach provides more control and flexibility over the data flow and analysis process.

label: `feature`

cc : @atris @kishoreg @Aravind-Suresh

Related Slack Discussion : https://apache-pinot.slack.com/archives/CDRCA57FC/p1687701635253309

Contributor guide

Open the contributing guide

Research direction

The issue proposes WebSocket query endpoints in the Pinot Broker, alongside the existing REST query API. Start by locating the Broker's current REST query endpoint implementation and review the linked Slack discussion to understand the intended protocol and requirements. Done means an agreed WebSocket design and implementation that supports the described low-latency streaming-query use case.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.