[Feature] add more realtime data sources
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
Since we disabled meetup stream in https://github.com/apache/pinot/pull/8183 , we currently only have github event stream as our only quickstart with real data flow
Propose to add more to the mix. Some of the great ones out there:
- [WikiEdit](https://en.wikipedia.org/w/api.php?action=help&modules=mergehistory) and other Wikipedia API stream
- [Twitter Edit](https://developer.twitter.com/en/docs/twitter-api)
Contributor guide
Research direction
Start by reviewing pull request 8183 and the existing GitHub event stream quickstart to understand how real data sources are configured. Compare the proposed Wikipedia and Twitter APIs, then define which additional source or sources are in scope; done should mean an additional working real-data quickstart is available.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, stream-processing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100