cloudpipe / cloudpipe/cloudpickle

Dynamic class reset state on every deserialization

Open
#439 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.9k
Forks
195
Avg merge
1d 10h
Merged PRs (30d)
1

Description

Reproducer:

```python
# Tested with cloudpickle 1.6.0
from cloudpickle import dumps, loads

class Klass:
classvar = None

def mutator():
Klass.classvar = 100

def check():
print("checking....")
print(f" Klass.classvar [{hex(id(Klass))}] = {Klass.classvar}")

def failing_case():
print("Klass", hex(id(Klass)))
saved = dumps(Klass)
mutator()
check()
loads(saved)
check()
loads(saved)
check()

if __name__ == '__main__':
failing_case()
```

Prints:

```
Klass 0x7fc698719980
checking....
Klass.classvar [0x7fc698719980] = 100
checking....
Klass.classvar [0x7fc698719980] = None
checking....
Klass.classvar [0x7fc698719980] = None
```

After each `loads(saved)`, the state in `Klass` is being reset unexpectedly.

This problem can appear like a tricky race condition in distributed, multi-threaded framework, such as Dask. See example https://gist.github.com/sklam/98e7c98ce909e76a3fa7904754db7bd9.

I created a patch for this in the vendored cloudpickle in Numba: https://github.com/numba/numba/pull/7388. Please let me know if there will be problems with the way I am fixing it. If it is okay, I can submit the PR here.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.