carpentries-incubator / carpentries-incubator/lesson-parallel-python
update dask delayed chapter
- Dominant language
- Dockerfile
- Stars
- 11
- Forks
- 15
- Avg merge
- 3m
- Merged PRs (30d)
- 3
Description
We could motivate the introduction of `dask.delayed` better by also introducing `asyncio`, `concurrent.futures` and `dask.futures` shortly. The advantage of `dask.delayed` is that we can visualize the call graph easily, and make visible what is happening. What all of these ideas have in common, is that they operate in a universe of "coloured" functions. If you want to call a delayed function, you receive a delayed object or future, if you want to unwrap this object, you have two choices, make the new function also delayed or run the computation (unwrapping). In this sense there is a sort of viral nature to all of these.
Still there are some major differences: `dask.delayed` is lazy, while most future implementations are eager; `concurrent.futures` runs using an `Executor` which can be multi-threading or multi-processing, while `dask.futures` also run distributed. Futures can be cancelled.
This issue replaces #82
Contributor guide
Research direction
Start by locating the delayed chapter in the lesson and read the existing introduction to dask.delayed. Add a brief comparison with asyncio, concurrent.futures, and dask.futures, covering laziness, execution models, distribution, cancellation, and call-graph visualization; the chapter is done when these differences and shared future-like behavior are clear.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100