Using Dask serialization in more cases
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
Looking through the code we can find a few examples of [functions with arguments and keyword arguments serialized with cloudpickle]( https://blog.dask.org/2019/06/20/load-image-data ). AFAICT this is used in [the `Client`]( https://github.com/dask/distributed/blob/5172678da28130ef7f4d86394d30566abc99328e/distributed/client.py#L2401-L2411 ) and [the `Worker`s]( https://github.com/dask/distributed/blob/df4206365bbb26196f7349b89bc7eccf3cc5de80/distributed/worker.py#L3278-L3292 ). Though maybe it is used in other places as well. In these cases it would be nice to rely on the serialization mechanisms of the underlying `Comm` involved instead of simply pickling.
Contributor guide
Assessment
This issue has not been assessed yet.