Maintain task graph on Client
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
Currently when the user submits jobs to the scheduler it sends up little sub-graphs, stores Futures for the results, and forgets the original computation. Instead, we could maintain an active `.dask` attribute on the Client. This would provide the following benefits:
1. It would be useful if the scheduler goes down and comes back up. We can resubmit all of the computations that we care about.
2. It would allow for local debugging. If a task were to fail we could bring the necessary data to the client and rerun the affected task, allowing for local `pdb` style debugging.
3. It would be useful for general understanding among expert users.
However, this isn't free. We would need to add some code to all of the task submission functions (`submit`, `map`, `compute`, `persist`) though this can probably be done just with the `Client._graph_to_futures` method. We would also need to add code on garbage collection.
My main motivation for this is to help address the High Availability question (benefit 1 above).
Contributor guide
Assessment
This issue has not been assessed yet.