TimelyDataflow / TimelyDataflow/timely-dataflow
Fault tolerance and availability
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 293
- Avg merge
- 14h 46m
- Merged PRs (30d)
- 4
Description
In the Naiad paper, we learn that:
Naiad has a simple but extensible implementation of fault tolerance: each stateful vertex implements a
CHECKPOINTandRESTOREinterface, and the system invokes these as appropriate to produce a consistent checkpoint across all workers.
Does Timely Dataflow also have this mechanism? Does it have an alternative? Does it not need it? Might it one day get it? Can it be implemented by users?
Sorry if these are silly questions. I'm only just learning about Naiad and Timely.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the linked Naiad paper and reviewing Timely Dataflow's current fault-tolerance and state-management entry points. The issue does not identify files, tests, or a defined change; completion would require deciding whether checkpoint and restore support is needed, what alternative exists, and what an implementable scope would be.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100