Web Socket Endpoints for querying in Pinot Broker
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 2d 55m
- Merged PRs (30d)
- 182
Description
Currently, we have REST API endpoints in pinot broker for querying. However, adding a WebSocket endpoint helps with having low latency & high throughput queries once the connection is established between the pinot broker and the client service.
**Use Case:**
Using WebSocket to connect to a Pinot table that is being populated from a Kafka topic / some streaming source. By not directly connecting to the Kafka topic, we can take advantage of data manipulation and efficient filtering, and aggregation features in Pinot. This approach provides more control and flexibility over the data flow and analysis process.
label: `feature`
cc : @atris @kishoreg @Aravind-Suresh
Related Slack Discussion : https://apache-pinot.slack.com/archives/CDRCA57FC/p1687701635253309
Contributor guide
Research direction
The issue proposes WebSocket query endpoints in the Pinot Broker, alongside the existing REST query API. Start by locating the Broker's current REST query endpoint implementation and review the linked Slack discussion to understand the intended protocol and requirements. Done means an agreed WebSocket design and implementation that supports the described low-latency streaming-query use case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- api, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100