dask / dask/distributed

Logger stalls

Open
#1,724 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

I'm getting odd errors in Python 3.6 when I create and destroy clusters in rapid succession.

```python
File "/home/mrocklin/workspace/distributed/distributed/worker.py", line 388, in _start
logger.info('start worker %d', id(self))
File "/home/mrocklin/Software/anaconda/lib/python3.6/logging/__init__.py", line 1301, in info
self._log(INFO, msg, args, **kwargs)
File "/home/mrocklin/Software/anaconda/lib/python3.6/logging/__init__.py", line 1437, in _log
self.handle(record)
File "/home/mrocklin/Software/anaconda/lib/python3.6/logging/__init__.py", line 1447, in handle
self.callHandlers(record)
File "/home/mrocklin/Software/anaconda/lib/python3.6/logging/__init__.py", line 1509, in callHandlers
hdlr.handle(record)
File "/home/mrocklin/Software/anaconda/lib/python3.6/logging/__init__.py", line 856, in handle
self.acquire()
File "/home/mrocklin/Software/anaconda/lib/python3.6/logging/__init__.py", line 807, in acquire
self.lock.acquire()
```
I've tried calling the `reset_logger_locks` as in https://github.com/dask/distributed/issues/1491 but then get the following:

```python
File "/home/mrocklin/workspace/distributed/distributed/nanny.py", line 519, in run
yield worker._start(*worker_start_args)
File "/home/mrocklin/workspace/tornado/tornado/gen.py", line 307, in wrapper
yielded = next(result)
File "/home/mrocklin/workspace/distributed/distributed/worker.py", line 379, in _start
logger.info(' Start worker at: %26s', self.address)
File "/home/mrocklin/Software/anaconda/lib/python3.6/logging/__init__.py", line 1301, in info
self._log(INFO, msg, args, **kwargs)
File "/home/mrocklin/Software/anaconda/lib/python3.6/logging/__init__.py", line 1437, in _log
self.handle(record)
File "/home/mrocklin/Software/anaconda/lib/python3.6/logging/__init__.py", line 1447, in handle
self.callHandlers(record)
File "/home/mrocklin/Software/anaconda/lib/python3.6/logging/__init__.py", line 1509, in callHandlers
hdlr.handle(record)
File "/home/mrocklin/Software/anaconda/lib/python3.6/logging/__init__.py", line 858, in handle
self.emit(record)
File "/home/mrocklin/Software/anaconda/lib/python3.6/logging/__init__.py", line 990, in emit
stream.write(self.terminator)
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.