cloudpipe / cloudpipe/cloudpickle

Cannot pickle Mock or MagicMock

Open
#457 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.9k
Forks
195
Avg merge
1d 10h
Merged PRs (30d)
1

Description

# Scenario

It happens that in tests somewhere deep in the object hierarchies and structures something has to be mocked. This especially happens with external services and their clients.

# Problem

If `cloudpickle` happens to stumble upon a mock, it fails and it's very difficult to track down which object and which attribute contains the mocked object (especially when there are a bunch of mocks).

# Example

**requirements.txt**
cloudpickle==2.0.0

**test_pickle.py**
```py
import unittest
from unittest.mock import MagicMock

import cloudpickle

class ToMock:
a: str

def __init__(self):
self.something = "something"

class Mockable:

def __init__(self, to_mock=None):
self.not_mocked = "this is not mocked"
self.to_be_mocked = to_mock or ToMock()

class TestPickling(unittest.TestCase):

def test_no_mocking(self):
self.assertIsInstance(cloudpickle.dumps(Mockable()), bytes)

def test_simple_mock(self):
self.assertIsInstance(cloudpickle.dumps(Mockable(MagicMock())), bytes)

def test_specced_mock(self):
self.assertIsInstance(cloudpickle.dumps(Mockable(MagicMock(spec=ToMock))), bytes)

def test_autospecced_mock(self):
self.assertIsInstance(cloudpickle.dumps(Mockable(unittest.mock.create_autospec(ToMock))), bytes)

if __name__ == '__main__':
unittest.main()
```

**output**
```
python test_pickle.py
Testing started at 19:05 ...
Launching unittests with arguments python -m unittest test_pickle.TestPickling in /home/bumbum/PycharmProjects/pythonProject

Error
Traceback (most recent call last):
File "/home/bumbum/PycharmProjects/pythonProject/test_pickle.py", line 33, in test_autospecced_mock
self.assertIsInstance(cloudpickle.dumps(Mockable(unittest.mock.create_autospec(ToMock))), bytes)
File "/home/python/cloudpickle-magic-mock/lib/python3.9/site-packages/cloudpickle/cloudpickle_fast.py", line 73, in dumps
cp.dump(obj)
File "/home/python/cloudpickle-magic-mock/lib/python3.9/site-packages/cloudpickle/cloudpickle_fast.py", line 602, in dump
return Pickler.dump(self, obj)
_pickle.PicklingError: args[0] from __newobj__ args has the wrong class

Error
Traceback (most recent call last):
File "/home/python/cloudpickle-magic-mock/lib/python3.9/site-packages/cloudpickle/cloudpickle_fast.py", line 602, in dump
return Pickler.dump(self, obj)
File "/usr/lib64/python3.9/unittest/mock.py", line 2500, in __call__
return _Call((self._mock_name, args, kwargs), name=name, parent=self)
File "/usr/lib64/python3.9/unittest/mock.py", line 2404, in __new__
_len = len(value)
RecursionError: maximum recursion depth exceeded while calling a Python object

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/bumbum/PycharmProjects/pythonProject/test_pickle.py", line 27, in test_simple_mock
self.assertIsInstance(cloudpickle.dumps(Mockable(MagicMock())), bytes)
File "/home/python/cloudpickle-magic-mock/lib/python3.9/site-packages/cloudpickle/cloudpickle_fast.py", line 73, in dumps
cp.dump(obj)
File "/home/python/cloudpickle-magic-mock/lib/python3.9/site-packages/cloudpickle/cloudpickle_fast.py", line 609, in dump
raise pickle.PicklingError(msg) from e
_pickle.PicklingError: Could not pickle object as excessively deep recursion required.

Error
Traceback (most recent call last):
File "/home/bumbum/PycharmProjects/pythonProject/test_pickle.py", line 30, in test_specced_mock
self.assertIsInstance(cloudpickle.dumps(Mockable(MagicMock(spec=ToMock))), bytes)
File "/home/python/cloudpickle-magic-mock/lib/python3.9/site-packages/cloudpickle/cloudpickle_fast.py", line 73, in dumps
cp.dump(obj)
File "/home/python/cloudpickle-magic-mock/lib/python3.9/site-packages/cloudpickle/cloudpickle_fast.py", line 602, in dump
return Pickler.dump(self, obj)
_pickle.PicklingError: args[0] from __newobj__ args has the wrong class

Ran 4 tests in 0.043s

FAILED (errors=3)
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.