Error involving cpu key in scheduler
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
I see that this error has been reported in: #3147 as well.
This is the error I'm seeing from running the scheduler.
```
distributed.utils - ERROR - 'cpu'
Traceback (most recent call last):
File "/nix/store/8zmsa97jhnns0p96ajqq20lan2hxbh7r-python3.7-distributed-2.3.0/lib/python3.7/site-packages/distributed/utils.py", line 657, in log_errors
yield
File "/nix/store/8zmsa97jhnns0p96ajqq20lan2hxbh7r-python3.7-distributed-2.3.0/lib/python3.7/site-packages/distributed/dashboard/scheduler.py", line 433, in update
cpu = [int(ws.metrics["cpu"]) for ws in workers]
```
This is how to reproduce this...
This is the scheduler.py.
```py
#!/usr/bin/env python3
import argparse
import asyncio
from dask.distributed import Scheduler
async def loop(host, port, dashboard_address):
async with Scheduler(
host=host, port=port, dashboard_address=dashboard_address
) as s:
await s.finished()
def main():
options_parser = argparse.ArgumentParser()
options_parser.add_argument("--host")
options_parser.add_argument("--port", type=int)
options_parser.add_argument("--dashboard-address")
options = options_parser.parse_args()
asyncio.get_event_loop().run_until_complete(
loop(
host=options.host,
port=options.port,
dashboard_address=options.dashboard_address,
)
)
if __name__ == "__main__":
main()
```
This is the worker.py
```py
#!/usr/bin/env python3
import argparse
import asyncio
from dask.distributed import Scheduler
async def loop(host, port, dashboard_address):
async with Scheduler(
host=host, port=port, dashboard_address=dashboard_address
) as s:
await s.finished()
def main():
options_parser = argparse.ArgumentParser()
options_parser.add_argument("--host")
options_parser.add_argument("--port", type=int)
options_parser.add_argument("--dashboard-address")
options = options_parser.parse_args()
asyncio.get_event_loop().run_until_complete(
loop(
host=options.host,
port=options.port,
dashboard_address=options.dashboard_address,
)
)
if __name__ == "__main__":
main()
```
```
./scheduler.py --host 127.0.0.1 --port 3201 --dashboard-address :3202
./worker.py --scheduler-ip 127.0.0.1 --scheduler-port 3201 --name image-classifier-worker
./worker.py --scheduler-ip 127.0.0.1 --scheduler-port 3201 --name image-classifier-worker
```
On the second invocation of the worker I get:
```
distributed.nanny - INFO - Start Nanny at: 'tcp://127.0.0.1:43955'
distributed.diskutils - INFO - Found stale lock file and directory '/home/cmcdragonkai/Projects/image-classifier/dask-worker-space/worker-xl3_qtz_', purging
distributed.worker - INFO - Start worker at: tcp://127.0.0.1:46821
distributed.worker - INFO - Listening to: tcp://127.0.0.1:46821
distributed.worker - INFO - Waiting to connect to: tcp://127.0.0.1:3201
distributed.worker - INFO - -------------------------------------------------
distributed.worker - INFO - Threads: 16
distributed.worker - INFO - Memory: 33.72 GB
distributed.worker - INFO - Local Directory: /home/cmcdragonkai/Projects/image-classifier/dask-worker-space/worker-xmi581os
distributed.worker - INFO - -------------------------------------------------
distributed.nanny - ERROR - Failed to start worker
Traceback (most recent call last):
File "/nix/store/8zmsa97jhnns0p96ajqq20lan2hxbh7r-python3.7-distributed-2.3.0/lib/python3.7/site-packages/distributed/nanny.py", line 674, in run
await worker
File "/nix/store/8zmsa97jhnns0p96ajqq20lan2hxbh7r-python3.7-distributed-2.3.0/lib/python3.7/site-packages/distributed/worker.py", line 1016, in start
await self._register_with_scheduler()
File "/nix/store/8zmsa97jhnns0p96ajqq20lan2hxbh7r-python3.7-distributed-2.3.0/lib/python3.7/site-packages/distributed/worker.py", line 835, in _register_with_scheduler
raise ValueError("Unexpected response from register: %r" % (response,))
ValueError: Unexpected response from register: {'status': 'error', 'message': 'name taken, image-classifier-worker', 'time': 1578361153.2849612}
distributed.comm.tcp - WARNING - Closing dangling stream in
distributed.nanny - INFO - Closing Nanny at 'tcp://127.0.0.1:43955'
```
But on the scheduler it starts having a failure regarding the `cpu` key.
Contributor guide
Assessment
This issue has not been assessed yet.