cloudpipe / cloudpipe/cloudpickle

cloudpickle pickle 'CudnnModule' object

Open
#405 6 comments 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.9k
Forks
195
Avg merge
1d 10h
Merged PRs (30d)
1

Description

I was spawning parallel processes using Joblib for training distributed PyTorch code, and Joblib uses cloudpickle to do its pickling. I've narrowed down the problem to cloudpickle, as demonstrated in the minimal repro below:

```python
# main.py
import torch.backends.cudnn
import cloudpickle

def foo():
print(torch.backends.cudnn)

cloudpickle.dumps(foo)
```
```shell
>>> python main.py

Traceback (most recent call last):
File "main.py", line 9, in
cloudpickle.dumps(foo)
File "/Users/BrianKo/anaconda3/envs/hydra38/lib/python3.8/site-packages/cloudpickle/cloudpickle_fast.py", line 73, in dumps
cp.dump(obj)
File "/Users/BrianKo/anaconda3/envs/hydra38/lib/python3.8/site-packages/cloudpickle/cloudpickle_fast.py", line 563, in dump
return Pickler.dump(self, obj)
TypeError: cannot pickle 'CudnnModule' object
```

The above script works when I use Python's pickle library instead. Is there a way to fix this?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.