cloudpipe / cloudpipe/cloudpickle
cloudpickle pickle 'CudnnModule' object
- Dominant language
- Python
- Stars
- 1.9k
- Forks
- 195
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 1
Description
I was spawning parallel processes using Joblib for training distributed PyTorch code, and Joblib uses cloudpickle to do its pickling. I've narrowed down the problem to cloudpickle, as demonstrated in the minimal repro below:
```python
# main.py
import torch.backends.cudnn
import cloudpickle
def foo():
print(torch.backends.cudnn)
cloudpickle.dumps(foo)
```
```shell
>>> python main.py
Traceback (most recent call last):
File "main.py", line 9, in
cloudpickle.dumps(foo)
File "/Users/BrianKo/anaconda3/envs/hydra38/lib/python3.8/site-packages/cloudpickle/cloudpickle_fast.py", line 73, in dumps
cp.dump(obj)
File "/Users/BrianKo/anaconda3/envs/hydra38/lib/python3.8/site-packages/cloudpickle/cloudpickle_fast.py", line 563, in dump
return Pickler.dump(self, obj)
TypeError: cannot pickle 'CudnnModule' object
```
The above script works when I use Python's pickle library instead. Is there a way to fix this?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.