dask / dask/distributed

Remaining pickle loads

Open
#2,051 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

Pickle can execute arbitrary code when deserialising. Therefore, some users may wish to disallow it as a serialisation mechanism, see https://github.com/dask/distributed/pull/1912 and follow-up PRs. Here I am listing remaining occurrences that might be insecure.

- pickle is referenced in functions `protocol.serialize.pickle_loads` and `protocol.pickle.loads`; the former is the one created for the selective deserialization mechanism
- `protocol.serialize.deserialize` is called in lots of places without specifying allowed deserializers (optional third argument). Some of these are necessarily safe
- `Serialize.deserialize`, I think this could be a problem
- deserialize_bytes should be OK, we know the data is a bytes object
- nested_deserialize, calls Serialize.deserialize
- specific functions for keras, netcdf, sparse; I doubt any python functions can be buried in these
- protocol.numpy.deserialize_numpy_ndarray; is it possible to formulate an array's dtype to contain code?
- client.Future.set_error, although errors are passed through core.error_message, I'm not certain they are clean. https://github.com/dask/distributed/pull/2016 was aimed at this, but the bare calls to loads are still here
- scheduler.Scheduler.feed , explicitly calls out that it is dangerous
- workers use loads for tasks and `run()`; I think he we care about the client side. In any case, a worker is not useful if we cannot deserialise functions, and the point is to execute code
- client.Client._handle_key_in_memory, for the "type" of a returned future; I guess it's possible to make a result object whose `type()` is itself a pickleable but dangerous object.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.