dask / dask/dask-cloudprovider

Fetch GCP project from the gcloud config when available

Open
#228 2 comments 0 reactions 0 assignees View on GitHub
enhancement provider/gcp/vm
Dominant language
Python
Stars
147
Forks
119
PR merge metrics
No merged PRs in 30d

Description

As per [documentation](https://cloudprovider.dask.org/en/latest/gcp.html#project-id) and error below setting project id is required:

```
>>> from dask_cloudprovider.gcp import GCPCluster
>>> cluster = GCPCluster(n_workers=1, machine_type="n1-standard-4")
Launching cluster with the following configuration:
Source Image: projects/ubuntu-os-cloud/global/images/ubuntu-minimal-1804-bionic-v20201014
Docker Image: daskdev/dask:latest
Machine Type: n1-standard-4
Filesytsem Size: 50
N-GPU Type:
Zone: us-east1-c
Creating scheduler instance
Traceback (most recent call last):
File "", line 1, in
File "/Users/rav/miniconda3/envs/dask_cloud/lib/python3.8/site-packages/dask_cloudprovider/gcp/instances.py", line 546, in __init__
super().__init__(**kwargs)
File "/Users/rav/miniconda3/envs/dask_cloud/lib/python3.8/site-packages/dask_cloudprovider/generic/vmcluster.py", line 235, in __init__
super().__init__(**kwargs)
File "/Users/rav/miniconda3/envs/dask_cloud/lib/python3.8/site-packages/distributed/deploy/spec.py", line 276, in __init__
self.sync(self._start)
... TRUNCATED ...
raise TypeError(
TypeError: Parameter "project" value "" does not match the pattern "(?:(?:[-a-z0-9]{1,63}\.)*(?:[a-z](?:[-a-z0-9]{0,61}[a-z0-9])?):)?(?:[0-9]{1,19}|(?:[a-z0-9](?:[-a-z0-9]{0,61}[a-z0-9])?))"
```

In case of missing project id and `gcloud` available maybe we could fetch the project from the gcloud config?

```bash
> gcloud config get-value project
foobar-project-id
```

Another option is to also fetch the project from the metadata service whilst running in GCE:

```bash
> curl "http://metadata.google.internal/computeMetadata/v1/project/project-id" -H "Metadata-Flavor: Google"
foobar-project-id
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.