dask / dask/distributed

Propagate environment variables to workers

Open
#2,908 14 comments 7 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

It is common place to use environment variables to configure security credentials such as AWS keys. When spawning a remote cluster, for example using `SSHCluster` or other remote `SpecCluster` derivatives it could be useful to propagate some or all environment variables from the parent Python process.

This could be implemented as a flag and would build on the work in #2508.

It could be a boolean

```python
with SSHCluster(..., propagate_env=True) as cluster:
...
```

or a list of envs to propagate

```python
with SSHCluster(..., propagate_env=['AWS_ACCESS_KEY_ID', 'AWS_SECRET_ACCESS_KEY', 'AWS_REGION']) as cluster:
...
```

or something else...

Thoughts?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.