GCSMap: error first time command is run but not the second

Open
#117 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
30/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
google-cloud, python
Domain
cloud

Research direction

The report identifies gcsfs.GCSMap and gcsfs/core.py:_call around line 458; first reproduce the failure with the exact snippet and inspect the authentication refresh path shown in the traceback. Done means the initial request succeeds consistently, or the expected first-call behavior is established with a regression test.

Written by the indexing model from the issue text.

Description

I'm trying to access data from a public GCS bucket using gcsfs.GCSMap.

The first time I run this code

import gcsfs
gcmap = gcsfs.GCSMap('pangeo-data/pyqg/barotropic/beta_00.zarr')

it fails with this error:

_call exception: ('invalid_grant: Bad Request', '{\n  "error": "invalid_grant",\n  "error_description": "Bad Request"\n}')
Traceback (most recent call last):
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/gcsfs/core.py", line 458, in _call
    r = meth(self.base + path, params=kwargs, json=json)
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/requests/sessions.py", line 537, in get
    return self.request('GET', url, **kwargs)
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/google/auth/transport/requests.py", line 183, in request
    self._auth_request, method, url, request_headers)
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/google/auth/credentials.py", line 121, in before_request
    self.refresh(request)
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/google/oauth2/credentials.py", line 117, in refresh
    self._client_secret))
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/google/oauth2/_client.py", line 191, in refresh_grant
    response_data = _token_endpoint_request(request, token_uri, body)
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/google/oauth2/_client.py", line 110, in _token_endpoint_request
    _handle_error_response(response_body)
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/google/oauth2/_client.py", line 60, in _handle_error_response
    error_details, response_body)
google.auth.exceptions.RefreshError: ('invalid_grant: Bad Request', '{\n  "error": "invalid_grant",\n  "error_description": "Bad Request"\n}')
_call exception: ('invalid_grant: Bad Request', '{\n  "error": "invalid_grant",\n  "error_description": "Bad Request"\n}')
Traceback (most recent call last):
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/gcsfs/core.py", line 458, in _call
    r = meth(self.base + path, params=kwargs, json=json)
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/requests/sessions.py", line 537, in get
    return self.request('GET', url, **kwargs)
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/google/auth/transport/requests.py", line 183, in request
    self._auth_request, method, url, request_headers)
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/google/auth/credentials.py", line 121, in before_request
    self.refresh(request)
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/google/oauth2/credentials.py", line 117, in refresh
    self._client_secret))
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/google/oauth2/_client.py", line 191, in refresh_grant
    response_data = _token_endpoint_request(request, token_uri, body)
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/google/oauth2/_client.py", line 110, in _token_endpoint_request
    _handle_error_response(response_body)
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/google/oauth2/_client.py", line 60, in _handle_error_response
    error_details, response_body)
google.auth.exceptions.RefreshError: ('invalid_grant: Bad Request', '{\n  "error": "invalid_grant",\n  "error_description": "Bad Request"\n}')
_call exception: ('invalid_grant: Bad Request', '{\n  "error": "invalid_grant",\n  "error_description": "Bad Request"\n}')
Traceback (most recent call last):
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/gcsfs/core.py", line 458, in _call
    r = meth(self.base + path, params=kwargs, json=json)
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/requests/sessions.py", line 537, in get
    return self.request('GET', url, **kwargs)
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/google/auth/transport/requests.py", line 183, in request
    self._auth_request, method, url, request_headers)
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/google/auth/credentials.py", line 121, in before_request
    self.refresh(request)
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/google/oauth2/credentials.py", line 117, in refresh
    self._client_secret))
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/google/oauth2/_client.py", line 191, in refresh_grant
    response_data = _token_endpoint_request(request, token_uri, body)
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/google/oauth2/_client.py", line 110, in _token_endpoint_request
    _handle_error_response(response_body)
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/google/oauth2/_client.py", line 60, in _handle_error_response
    error_details, response_body)
google.auth.exceptions.RefreshError: ('invalid_grant: Bad Request', '{\n  "error": "invalid_grant",\n  "error_description": "Bad Request"\n}')

If I run the exact same line again, it works. This is confusing. I remember a similar thing happened on a pangeo binder environment.

I'm using gcsfs version 0.1.2 installed from conda forge.

Dominant language
Python
Stars
398
Forks
181
Avg merge
2d 10h
Merged PRs (30d)
36

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from fsspec/gcsfs

All issues in fsspec/gcsfs

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.