Getting 'GS' key error when reading a csv from GCS using gcsfc
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 398
- Forks
- 181
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 36
Description
Hi I upgraded gcsfs and now I get the following error:
My code is pretty simple:
data = dd.read_csv(file_path, parse_dates=[date_column])\
.compute()
return data```
It used to work but all of a sudden it stopped working.
file_path = gs://mybuck/res.csv
```File "main.py", line 51, in run
data = load_parse_file(file_path=args.input_file)
File "/FbProphet/prophet_gcp/utils.py", line 15, in load_parse_file
data = dd.read_csv(file_path, parse_dates=[date_column])\
File "/work/miniconda/lib/python3.7/site-packages/dask/dataframe/io/csv.py", line 578, in read
**kwargs
File "/work/miniconda/lib/python3.7/site-packages/dask/dataframe/io/csv.py", line 405, in read_pandas
**(storage_options or {})
File "/work/miniconda/lib/python3.7/site-packages/dask/bytes/core.py", line 93, in read_bytes
fs, fs_token, paths = get_fs_token_paths(urlpath, mode="rb", storage_options=kwargs)
File "/work/miniconda/lib/python3.7/site-packages/dask/bytes/core.py", line 425, in get_fs_token_paths
fs, fs_token = get_fs(protocol, options)
File "/work/miniconda/lib/python3.7/site-packages/dask/bytes/core.py", line 571, in get_fs
cls = _filesystems[protocol]
KeyError: 'gs'
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the reported dask.dataframe.read_csv call and the traceback through dask/bytes/core.py, especially get_fs_token_paths and get_fs. Reproduce the gs:// read with the upgraded gcsfs setup and compare filesystem registration behavior; done means the CSV loads without KeyError: 'gs'.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, python
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100