TimelyDataflow / TimelyDataflow/timely-dataflow
How to use timely dataflow with realtime input data?
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 293
- Avg merge
- 14h 46m
- Merged PRs (30d)
- 4
Description
I m quite new to timely dataflow, but I have some experience in Hadoop and Spark. My question is how can I simulate realtime input data and plug into timely dataflow. Something like this Twitter demo in this video at 21:30 https://channel9.msdn.com/posts/Frank-McSherry-Introduction-to-Naiad-and-Differential-Dataflow. I cant find any example about this.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files, tests, or entry points. Start by locating the existing timely dataflow input and example code, then determine how a simulated realtime source should connect to a dataflow. Done means a documented, runnable example explains how to feed realtime-style input into timely dataflow.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- stream-processing
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100