dask / dask/distributed

Setting a basicConfig before importing distributed changes the scheduler logging level.

Open
#8,805 0 comments 0 reactions 0 assignees View on GitHub
needs triage
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

Hello,

Some kind of follow-up to the #7527 issue.

**Describe the issue**:

Setting a basicConfig **before the distributed import** changes the logging level of the scheduler to INFO.
Doing the same with the distributed import before the logging configuration does work as intended.

**Minimal Complete Verifiable Example**:

```python
import logging

DEFAULT_FORMAT = "%(asctime)s %(levelname)-7s %(message)s"
DEFAULT_TIME_FORMAT = "%Y-%m-%d %H:%M:%S"

logging.basicConfig(format=DEFAULT_FORMAT, datefmt=DEFAULT_TIME_FORMAT)

import distributed

loc_cluster = distributed.LocalCluster(
processes=False, n_workers=1, silence_logs="ERROR"
)
```

This code output the following logs:
```
2024-07-27 09:52:13 INFO To route to workers diagnostics web server please install jupyter-server-proxy: python -m pip install jupyter-server-proxy
2024-07-27 09:52:13 INFO State start
2024-07-27 09:52:13 INFO Scheduler at: inproc://10.40.206.22/2275623/1
2024-07-27 09:52:13 INFO dashboard at: http://10.40.206.22:8787/status
2024-07-27 09:52:13 INFO Registering Worker plugin shuffle
2024-07-27 09:52:13 INFO Start worker at: inproc://10.40.206.22/2275623/4
2024-07-27 09:52:13 INFO Listening to: inproc10.40.206.22
2024-07-27 09:52:13 INFO Worker name: 0
2024-07-27 09:52:13 INFO dashboard at: 10.40.206.22:41269
2024-07-27 09:52:13 INFO Waiting to connect to: inproc://10.40.206.22/2275623/1
2024-07-27 09:52:13 INFO -------------------------------------------------
2024-07-27 09:52:13 INFO Threads: 96
2024-07-27 09:52:13 INFO Memory: 629.52 GiB
2024-07-27 09:52:13 INFO Local Directory: /tmp/dask-scratch-space-10238/worker-frdknfmy
2024-07-27 09:52:13 INFO -------------------------------------------------
2024-07-27 09:52:13 INFO Register worker
2024-07-27 09:52:13 INFO Starting worker compute stream, inproc://10.40.206.22/2275623/4
2024-07-27 09:52:13 INFO Starting established connection to inproc://10.40.206.22/2275623/5
2024-07-27 09:52:13 INFO Starting Worker plugin shuffle
2024-07-27 09:52:13 INFO Registered to: inproc://10.40.206.22/2275623/1
2024-07-27 09:52:13 INFO -------------------------------------------------
2024-07-27 09:52:13 INFO Starting established connection to inproc://10.40.206.22/2275623/1
```

**Environment**:

- Dask version: 2024.7.1
- Python version: 3.12.4
- Operating System: Ubuntu 22.04
- Install method (conda, pip, source): conda

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.