Remaining pickle loads
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
Pickle can execute arbitrary code when deserialising. Therefore, some users may wish to disallow it as a serialisation mechanism, see https://github.com/dask/distributed/pull/1912 and follow-up PRs. Here I am listing remaining occurrences that might be insecure.
- pickle is referenced in functions `protocol.serialize.pickle_loads` and `protocol.pickle.loads`; the former is the one created for the selective deserialization mechanism
- `protocol.serialize.deserialize` is called in lots of places without specifying allowed deserializers (optional third argument). Some of these are necessarily safe
- `Serialize.deserialize`, I think this could be a problem
- deserialize_bytes should be OK, we know the data is a bytes object
- nested_deserialize, calls Serialize.deserialize
- specific functions for keras, netcdf, sparse; I doubt any python functions can be buried in these
- protocol.numpy.deserialize_numpy_ndarray; is it possible to formulate an array's dtype to contain code?
- client.Future.set_error, although errors are passed through core.error_message, I'm not certain they are clean. https://github.com/dask/distributed/pull/2016 was aimed at this, but the bare calls to loads are still here
- scheduler.Scheduler.feed , explicitly calls out that it is dangerous
- workers use loads for tasks and `run()`; I think he we care about the client side. In any case, a worker is not useful if we cannot deserialise functions, and the point is to execute code
- client.Client._handle_key_in_memory, for the "type" of a returned future; I guess it's possible to make a result object whose `type()` is itself a pickleable but dangerous object.
Contributor guide
Assessment
This issue has not been assessed yet.