Logging configuration using environment variables
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
**Describe the issue**:
The logging configuration can not be set using environment variables. However, it is the easiest way to pass some configuration to the cluster's workers. Specifically, I would like to update the following settings:
```python
dask.config.get("distributed.logging")
```
```
{'distributed': 'debug', 'distributed.client': 'debug', 'distributed.scheduler': 'debug', 'distributed.shuffle': 'debug'}
```
**Minimal Complete Verifiable Example**:
The following code throws an error
```python
import os
os.environ['DASK_DISTRIBUTED__LOGGING__DISTRIBUTED'] = 'info'
os.environ['DASK_DISTRIBUTED__LOGGING__DISTRIBUTED__CLIENT'] = 'info'
os.environ['DASK_DISTRIBUTED__LOGGING__DISTRIBUTED__SCHEDULER'] = 'info'
os.environ['DASK_DISTRIBUTED__LOGGING__DISTRIBUTED__SHUFFLE'] = 'info'
import dask
import dask.distributed
```
Trace:
```text
Traceback (most recent call last):
File ".../test.py", line 8, in
import dask.distributed
File ".../.venv/lib/python3.10/site-packages/dask/__init__.py", line 3, in
from dask import config, datasets
File ".../.venv/lib/python3.10/site-packages/dask/config.py", line 802, in
refresh()
File ".../.venv/lib/python3.10/site-packages/dask/config.py", line 522, in refresh
update(config, collect(**kwargs))
File ".../.venv/lib/python3.10/site-packages/dask/config.py", line 485, in collect
configs.append(collect_env(env=env))
File ".../.venv/lib/python3.10/site-packages/dask/config.py", line 270, in collect_env
set(d, config=result)
File ".../.venv/lib/python3.10/site-packages/dask/config.py", line 391, in __init__
self._assign(key.split("."), value, config)
File ".../.venv/lib/python3.10/site-packages/dask/config.py", line 458, in _assign
self._assign(keys[1:], value, d[key], path, record=record)
File ".../.venv/lib/python3.10/site-packages/dask/config.py", line 458, in _assign
self._assign(keys[1:], value, d[key], path, record=record)
File ".../.venv/lib/python3.10/site-packages/dask/config.py", line 458, in _assign
self._assign(keys[1:], value, d[key], path, record=record)
File ".../.venv/lib/python3.10/site-packages/dask/config.py", line 450, in _assign
d[key] = value
TypeError: 'str' object does not support item assignment
```
**Environment**:
- Dask version: 2024.1.0
- Python version: 3.10
- Operating System: Win11, WSL
- Install method (conda, pip, source): poetry
Contributor guide
Assessment
This issue has not been assessed yet.