Dask.destributed Client() specific - Could not serialize object of type DataFrame
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
Hello All,
I am consistently getting the following error when I include:
```
cluster = LocalCluster()
client = Client(cluster)
```
If I run the same code without using the Client() at all (aka none of the above code), the task runs fine. However, I would prefer to use the above scheduler, as it seems that not including the above code results in a large amount of leaked memory which, over times causes the memory usage to be enormous, which based on the script, it should not be. From looking around a ton of other issues, it looks like this memory leak is a well known issue with the default scheduler, but is commonly resolved with the Client() scheduler. Any Ideas on how to resolve the below error?
Note, I did update all the packages (Dask and its dependents) using pip earlier today and I am still getting the error. Also note that Client(processes = False) works without issue, however I would very much prefer (processes = True) for the performance gain.
```
distributed.protocol.core - CRITICAL - Failed to Serialize
Traceback (most recent call last):
File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/distributed/protocol/core.py", line 53, in dumps
for key, value in data.items()
File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/distributed/protocol/core.py", line 54, in
if type(value) is Serialize}
File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/distributed/protocol/serialize.py", line 157, in serialize
raise TypeError(msg)
TypeError: Could not serialize object of type DataFrame
distributed.comm.utils - ERROR - Could not serialize object of type DataFrame
Traceback (most recent call last):
File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/distributed/comm/utils.py", line 40, in _to_frames
context=context))
File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/distributed/protocol/core.py", line 53, in dumps
for key, value in data.items()
File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/distributed/protocol/core.py", line 54, in
if type(value) is Serialize}
File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/distributed/protocol/serialize.py", line 157, in serialize
raise TypeError(msg)
TypeError: Could not serialize object of type DataFrame
distributed.batched - ERROR - Error in batched write
Traceback (most recent call last):
File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/distributed/batched.py", line 94, in _background_send
on_error='raise')
File "/home/sholt/.local/lib/python2.7/site-packages/tornado/gen.py", line 1099, in run
value = future.result()
File "/home/sholt/.local/lib/python2.7/site-packages/tornado/concurrent.py", line 260, in result
raise_exc_info(self._exc_info)
File "/home/sholt/.local/lib/python2.7/site-packages/tornado/gen.py", line 1107, in run
yielded = self.gen.throw(*exc_info)
File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/distributed/comm/tcp.py", line 221, in write
'recipient': self._peer_addr})
File "/home/sholt/.local/lib/python2.7/site-packages/tornado/gen.py", line 1099, in run
value = future.result()
File "/home/sholt/.local/lib/python2.7/site-packages/tornado/concurrent.py", line 260, in result
raise_exc_info(self._exc_info)
File "/home/sholt/.local/lib/python2.7/site-packages/tornado/gen.py", line 315, in wrapper
yielded = next(result)
File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/distributed/comm/utils.py", line 49, in to_frames
res = _to_frames()
File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/distributed/comm/utils.py", line 40, in _to_frames
context=context))
File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/distributed/protocol/core.py", line 53, in dumps
for key, value in data.items()
File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/distributed/protocol/core.py", line 54, in
if type(value) is Serialize}
File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/distributed/protocol/serialize.py", line 157, in serialize
raise TypeError(msg)
TypeError: Could not serialize object of type DataFrame
```
Contributor guide
Assessment
This issue has not been assessed yet.