cloudpipe / cloudpipe/cloudpickle
Dynamic class reset state on every deserialization
- Dominant language
- Python
- Stars
- 1.9k
- Forks
- 195
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 1
Description
Reproducer:
```python
# Tested with cloudpickle 1.6.0
from cloudpickle import dumps, loads
class Klass:
classvar = None
def mutator():
Klass.classvar = 100
def check():
print("checking....")
print(f" Klass.classvar [{hex(id(Klass))}] = {Klass.classvar}")
def failing_case():
print("Klass", hex(id(Klass)))
saved = dumps(Klass)
mutator()
check()
loads(saved)
check()
loads(saved)
check()
if __name__ == '__main__':
failing_case()
```
Prints:
```
Klass 0x7fc698719980
checking....
Klass.classvar [0x7fc698719980] = 100
checking....
Klass.classvar [0x7fc698719980] = None
checking....
Klass.classvar [0x7fc698719980] = None
```
After each `loads(saved)`, the state in `Klass` is being reset unexpectedly.
This problem can appear like a tricky race condition in distributed, multi-threaded framework, such as Dask. See example https://gist.github.com/sklam/98e7c98ce909e76a3fa7904754db7bd9.
I created a patch for this in the vendored cloudpickle in Numba: https://github.com/numba/numba/pull/7388. Please let me know if there will be problems with the way I am fixing it. If it is okay, I can submit the PR here.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.