Cannot maintain effective debug-level on distributed (parent) logger with local scheduler/cluster
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
With `config.yaml` including simple-form `logging` config:
```
logging:
distributed: debug
>>> import logging
>>> import distributed
>>> logging.getLogger("distributed")
>>> logging.getLogger("distributed").handlers
[ (NOTSET)>]
>>> logging.getLogger("distributed").debug("debug")
distributed - DEBUG - debug
>>>
```
All looks good. Start a `Client`:
```
>>> from distributed import Client
>>> c = Client()
>>> logging.getLogger("distributed")
>>> logging.getLogger("distributed").handlers
[ (WARNING)>]
>>> logging.getLogger("distributed").debug("debug")
>>>
```
No logging output, as the `distributed` logger has had its `Handler` (a `StreamHandler`) set to `WARNING` threshold.
Once a `Client` is started and fires-up a local cluster/scheduler, I suspect a worker:
https://github.com/dask/distributed/blob/09b959a5667a51a2dc073510c784e01d44827457/distributed/worker.py#L424-L425
... runs `silence_logging`, which does just as it is named:
https://github.com/dask/distributed/blob/09b959a5667a51a2dc073510c784e01d44827457/distributed/utils.py#L728-L742
See also #2659 . It might be argued that nothing important is logging to the `distributed` logger, and everything is logging to a child e.g. `distributed.client` (even through the loggers are flattened with the simple config), which might be true, but it further erodes the notion of hierarchical logging.
Contributor guide
Assessment
This issue has not been assessed yet.