dask / dask/distributed

test_fail_write_to_disk fails on 32-bit systems

Open
#5,252 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

**What happened**:

When [running tests on a 32-bit system](https://koji.fedoraproject.org/koji/taskinfo?taskID=74314301), `test_fail_write_to_disk` fails:

```
___________________________ test_fail_write_to_disk ____________________________
c =
s =
a =
b =
@gen_cluster(client=True)
async def test_fail_write_to_disk(c, s, a, b):
class Bad:
def __getstate__(self):
raise TypeError()

def __sizeof__(self):
return int(100e9)

future = c.submit(Bad)
await wait(future)

> assert future.status == "error"
E AssertionError: assert 'finished' == 'error'
E - error
E + finished
distributed/tests/test_worker.py:905: AssertionError
----------------------------- Captured stderr call -----------------------------
distributed.scheduler - INFO - Clear task state
distributed.scheduler - INFO - Scheduler at: tcp://127.0.0.1:33725
distributed.scheduler - INFO - dashboard at: 127.0.0.1:38449
distributed.worker - INFO - Start worker at: tcp://127.0.0.1:39547
distributed.worker - INFO - Listening to: tcp://127.0.0.1:39547
distributed.worker - INFO - dashboard at: 127.0.0.1:41927
distributed.worker - INFO - Waiting to connect to: tcp://127.0.0.1:33725
distributed.worker - INFO - -------------------------------------------------
distributed.worker - INFO - Threads: 1
distributed.worker - INFO - Memory: 14.64 GiB
distributed.worker - INFO - Local Directory: /builddir/build/BUILD/distributed-2021.08.1/dask-worker-space/worker-1pquur4d
distributed.worker - INFO - -------------------------------------------------
distributed.worker - INFO - Start worker at: tcp://127.0.0.1:42475
distributed.worker - INFO - Listening to: tcp://127.0.0.1:42475
distributed.worker - INFO - dashboard at: 127.0.0.1:43523
distributed.worker - INFO - Waiting to connect to: tcp://127.0.0.1:33725
distributed.worker - INFO - -------------------------------------------------
distributed.worker - INFO - Threads: 2
distributed.worker - INFO - Memory: 14.64 GiB
distributed.worker - INFO - Local Directory: /builddir/build/BUILD/distributed-2021.08.1/dask-worker-space/worker-_cw23ys4
distributed.worker - INFO - -------------------------------------------------
distributed.scheduler - INFO - Register worker
distributed.scheduler - INFO - Starting worker compute stream, tcp://127.0.0.1:39547
distributed.core - INFO - Starting established connection
distributed.scheduler - INFO - Register worker
distributed.scheduler - INFO - Starting worker compute stream, tcp://127.0.0.1:42475
distributed.core - INFO - Starting established connection
distributed.worker - INFO - Registered to: tcp://127.0.0.1:33725
distributed.worker - INFO - -------------------------------------------------
distributed.worker - INFO - Registered to: tcp://127.0.0.1:33725
distributed.worker - INFO - -------------------------------------------------
distributed.core - INFO - Starting established connection
distributed.core - INFO - Starting established connection
distributed.scheduler - INFO - Receive client connection: Client-65de15cc-0314-11ec-bb55-525400769539
distributed.core - INFO - Starting established connection
distributed.sizeof - WARNING - Sizeof calculation failed. Defaulting to 1MB
Traceback (most recent call last):
File "/builddir/build/BUILD/distributed-2021.08.1/distributed/sizeof.py", line 14, in safe_sizeof
return sizeof(obj)
File "/usr/lib/python3.9/site-packages/dask/utils.py", line 575, in __call__
return meth(arg, *args, **kwargs)
File "/usr/lib/python3.9/site-packages/dask/sizeof.py", line 22, in sizeof_default
return getsizeof(o)
OverflowError: Python int too large to convert to C ssize_t
distributed.sizeof - WARNING - Sizeof calculation failed. Defaulting to 1MB
Traceback (most recent call last):
File "/builddir/build/BUILD/distributed-2021.08.1/distributed/sizeof.py", line 14, in safe_sizeof
return sizeof(obj)
File "/usr/lib/python3.9/site-packages/dask/utils.py", line 575, in __call__
return meth(arg, *args, **kwargs)
File "/usr/lib/python3.9/site-packages/dask/sizeof.py", line 22, in sizeof_default
return getsizeof(o)
OverflowError: Python int too large to convert to C ssize_t
distributed.sizeof - WARNING - Sizeof calculation failed. Defaulting to 1MB
Traceback (most recent call last):
File "/builddir/build/BUILD/distributed-2021.08.1/distributed/sizeof.py", line 14, in safe_sizeof
return sizeof(obj)
File "/usr/lib/python3.9/site-packages/dask/utils.py", line 575, in __call__
return meth(arg, *args, **kwargs)
File "/usr/lib/python3.9/site-packages/dask/sizeof.py", line 22, in sizeof_default
return getsizeof(o)
OverflowError: Python int too large to convert to C ssize_t
distributed.scheduler - INFO - Remove client Client-65de15cc-0314-11ec-bb55-525400769539
distributed.scheduler - INFO - Remove client Client-65de15cc-0314-11ec-bb55-525400769539
distributed.scheduler - INFO - Close client connection: Client-65de15cc-0314-11ec-bb55-525400769539
distributed.worker - INFO - Stopping worker at tcp://127.0.0.1:39547
distributed.worker - INFO - Stopping worker at tcp://127.0.0.1:42475
distributed.scheduler - INFO - Remove worker
distributed.core - INFO - Removing comms to tcp://127.0.0.1:39547
distributed.scheduler - INFO - Remove worker
distributed.core - INFO - Removing comms to tcp://127.0.0.1:42475
distributed.scheduler - INFO - Lost all workers
distributed.scheduler - INFO - Scheduler closing...
distributed.scheduler - INFO - Scheduler closing all comms
```

It appears that the `__sizeof__` overflows, and Dask then picks a default size that is so small that it doesn't exercise the problem that the test is trying to trigger.

**What you expected to happen**:

Tests pass.

**Minimal Complete Verifiable Example**:

```python
$ pytest -m 'not avoid_ci and not slow and not asyncio' -vra
```

**Environment**:

- Dask version: 2021.8.1
- Python version: 3.9.6
- Operating System: Fedora 34
- Install method (conda, pip, source): sources

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.