Workers not using environment variables for default config options
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
**WARNING**: The behavior reported here may **not** be a bug. For example, the consensus may be: "You must set your environment variables before importing dask." However, I want to document the recent change in behavior, because it's breaking dask-cuda CI.
**What happened**:
The workers within a `LocalCluster` do not use environment variables to set default config options unless the environment variables were set **before** `dask` was imported.
**What you expected to happen**:
Environment variables should still be used to set `dask.config` defaults after importing `dask`.
**Minimal Complete Verifiable Example**:
```python
import os
import dask
from dask.distributed import LocalCluster, Client
def check_config():
assert os.environ.get("DASK_DISTRIBUTED__COMM__COMPRESSION") == "False"
print(f"distributed.comm.compression={dask.config.get('distributed.comm.compression')}")
os.environ["DASK_DISTRIBUTED__COMM__COMPRESSION"] = "False"
with LocalCluster(n_workers=1) as cluster:
with Client(cluster) as client:
client.run(check_config)
# Prints: distributed.comm.compression=auto
```
Contributor guide
Assessment
This issue has not been assessed yet.