service account permissions error
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 398
- Forks
- 181
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 36
Description
I am trying to use gcsfs with a GCS service account .json token. I created a token at https://console.cloud.google.com/iam-admin/serviceaccounts/ and assigned it the role of "Storage Admin". This should have permissions to do anything to my GCS resources. I downloaded the .json token.
I use this with gcsfs as follows:
fs = gcsfs.GCSFileSystem(project='pangeo-181919', token='/home/rpa/pangeo-bf62fe06ed97.json')
fs.buckets
I get this error:
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
<ipython-input-14-81fa34b27c7f> in <module>()
1 # connect to gcs
2 fs = gcsfs.GCSFileSystem(project='pangeo-181919', token='/home/rpa/pangeo-bf62fe06ed97.json')
----> 3 fs.buckets
/home/rpa/.conda/envs/dask_distributed/lib/python3.5/site-packages/gcsfs/core.py in buckets(self)
449 def buckets(self):
450 """Return list of available project buckets."""
--> 451 return [b["name"] for b in self._list_buckets()["items"]]
452
453 @classmethod
<decorator-gen-128> in _list_buckets(self)
/home/rpa/.conda/envs/dask_distributed/lib/python3.5/site-packages/gcsfs/core.py in _tracemethod(f, self, *args, **kwargs)
49 logger.log(logging.DEBUG - 1, tb_io.getvalue())
50
---> 51 return f(self, *args, **kwargs)
52
53
/home/rpa/.conda/envs/dask_distributed/lib/python3.5/site-packages/gcsfs/core.py in _list_buckets(self)
568 items = []
569 page = self._call(
--> 570 'get', 'b/', project=self.project
571 )
572
<decorator-gen-123> in _call(self, method, path, *args, **kwargs)
/home/rpa/.conda/envs/dask_distributed/lib/python3.5/site-packages/gcsfs/core.py in _tracemethod(f, self, *args, **kwargs)
49 logger.log(logging.DEBUG - 1, tb_io.getvalue())
50
---> 51 return f(self, *args, **kwargs)
52
53
/home/rpa/.conda/envs/dask_distributed/lib/python3.5/site-packages/gcsfs/core.py in _call(self, method, path, *args, **kwargs)
430 time.sleep(2**retry - 1)
431 r = meth(self.base + path, params=kwargs, json=json)
--> 432 validate_response(r, path)
433 break
434 except (HtmlError, RequestException, GoogleAuthError) as e:
/home/rpa/.conda/envs/dask_distributed/lib/python3.5/site-packages/gcsfs/core.py in validate_response(r, path)
156 raise FileNotFoundError(path)
157 elif "forbidden" in m:
--> 158 raise IOError("Forbidden: %s\n%s" % (path, msg))
159 elif "invalid" in m:
160 raise ValueError("Bad Request: %s\n%s" % (path, msg))
OSError: Forbidden: b/
gcsfs-gyre-token@pangeo-181919.iam.gserviceaccount.com does not have storage.buckets.list access to project 464800473488.
This doesn't make sense to me. It seems like listing buckets should definitely be within the privileges of the storage admin.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the reported GCSFileSystem constructor and fs.buckets call, then inspect the _list_buckets, _call, and validate_response paths in gcsfs/core.py alongside the service-account role and project IDs shown in the report. Done means determining whether the forbidden response is expected for this configuration and documenting or correcting the confirmed issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, python
- Domain
- authorization, cloud
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100