apache / apache/pinot

Multi-Stream Consumption

Open
#7,514 2 comments 0 reactions 1 assignee Claimed by @sajjad-moradi View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 3h
Merged PRs (30d)
195

Description

Currently one realtime Pinot table only consumes from one stream. For use cases in which multiple existing streams are the input source for one Pinot table, the workaround is to use stream processors to read from different streams, perform necessary transformations, and write the output of all transformed events into another stream. Then one realtime pinot table can be set up to consume from that output stream. This requires maintaining two systems: the stream processor jobs and also pinot realtime tables. It would be nice to add capabilities to pinot realtime tables to consume from multiple topics.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.