dask / dask/dask-cloudprovider

Plugins that help to pass credentials for S3 and GCS to remote cluster workers

Open
#438 5 comments 0 reactions 0 assignees View on GitHub
provider/aws/ec2 provider/gcp/vm question
Dominant language
Python
Stars
147
Forks
119
PR merge metrics
No merged PRs in 30d

Description

I didn't find a simple way to pass credentials to remote workers, such as S3 and GCS, while both are widely used to store data frames.
In this ticket's scope, I propose creating plugins that will help distribute the required keys to remote workers.

**GCP credentials**
GCP credentials file path is stored in `GOOGLE_APPLICATION_CREDENTIALS` env variable. The plugin has to create a remote file and pass an env variable with a proper path to workers.

**S3 credentials**
Like GCP, we must update credential files and store them on each worker.

PR: #439

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.