cloudpipe / cloudpipe/cloudpickle

Inconsistent dumps of function in different contexts

Open
#168 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.9k
Forks
195
Avg merge
1d 10h
Merged PRs (30d)
1

Description

Pickling a function in the REPL and from the command line gives different results, it looks like the closure is not included in the REPL version. This results in problems, for example, when unpickling on remote workers such as dask.

```
$ cat cloudpickle_example.py
import cloudpickle
def some_function(a, b=1):
""" things """
return a, b

if __name__ == '__main__':
print(cloudpickle.dumps(some_function))
$ python cloudpickle_example.py
b'\x80\x04\x954\x01\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x0e_fill_function\x94\x93\x94(h\x00\x8c\x0f_make_skel_func\x94\x93\x94h\x00\x8c\r_builtin_type\x94\x93\x94\x8c\x08CodeType\x94\x85\x94R\x94(K\x02K\x00K\x02K\x02KCC\x08|\x00|\x01f\x02S\x00\x94\x8c\x08 things \x94\x85\x94)\x8c\x01a\x94\x8c\x01b\x94\x86\x94\x8c\x16cloudpickle_example.py\x94\x8c\rsome_function\x94K\x02C\x02\x00\x02\x94))t\x94R\x94J\xff\xff\xff\xff}\x94\x87\x94R\x94}\x94(\x8c\x07globals\x94}\x94\x8c\x08defaults\x94K\x01\x85\x94\x8c\x04dict\x94}\x94\x8c\x06module\x94\x8c\x08__main__\x94\x8c\x0eclosure_values\x94N\x8c\x08qualname\x94h\x11utR.'
$ ipython
Python 3.6.4 |Anaconda, Inc.| (default, Mar 12 2018, 20:05:31)
Type 'copyright', 'credits' or 'license' for more information
IPython 6.2.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import cloudpickle_example as c; import cloudpickle

In [2]: cloudpickle.dumps(c.some_function)
Out[2]: b'\x80\x04\x95)\x00\x00\x00\x00\x00\x00\x00\x8c\x13cloudpickle_example\x94\x8c\rsome_function\x94\x93\x94.'
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.