TimelyDataflow / TimelyDataflow/timely-dataflow
Comparison between Timely-Dataflow and Apache Flink
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 293
- Avg merge
- 14h 46m
- Merged PRs (30d)
- 4
Description
I have been exploring both Timely-DataFlow (TD) and Apache Flink (AF) for some graph analytics use cases and it seems that both the frameworks offer similar features (e.g. iterations, statefulness, streaming/batch computations etc..). Although Flink seems to be production ready, the performance of TD was impressive - Ran a PR algo on a LiveJournal dataset on macbook pro with 16GB RAM, with worker level aggregation (only one worker). TD finished in about 13 secs where as AF took about 220 secs. These are out of the box numbers.
Are there any features in TD that are missing in AF or vice-versa ? I am trying to figure out the main architectural differences.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
This is an architectural comparison request rather than a code task; the issue provides no file, test, or entry point to inspect. Begin with the reported PageRank timings and the stated feature overlap between Timely-Dataflow and Apache Flink. Done would be a clear comparison of their architectural differences and missing capabilities.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- distributed-systems
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100