fsspec / fsspec/gcsfs

FileNotFoundError on file.read

Open
#184 17 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
398
Forks
181
Avg merge
2d 10h
Merged PRs (30d)
36

Description

I have uploaded a file to GCS and was able to read it without problem. After that I deleted the file and uploaded it again with a same name. When I called file.read function again, I receive FileNotFoundError coming from fetch range though the file exists. Full traceback:

    byte_str = csv_file.read(4096)
  File "/opt/conda/default/lib/python3.6/site-packages/fsspec/spec.py", line 1040, in read
    out = self.cache._fetch(self.loc, self.loc + length)
  File "/opt/conda/default/lib/python3.6/site-packages/fsspec/core.py", line 464, in _fetch
    self.cache = self.fetcher(start, end + self.blocksize)
  File "</opt/conda/default/lib/python3.6/site-packages/decorator.py:decorator-gen-22>", line 2, in _fetch_range
  File "/opt/conda/default/lib/python3.6/site-packages/gcsfs/core.py", line 54, in _tracemethod
    return f(self, *args, **kwargs)
  File "/opt/conda/default/lib/python3.6/site-packages/gcsfs/core.py", line 1067, in _fetch_range
    headers=head)
  File "</opt/conda/default/lib/python3.6/site-packages/decorator.py:decorator-gen-2>", line 2, in _call
  File "/opt/conda/default/lib/python3.6/site-packages/gcsfs/core.py", line 54, in _tracemethod
    return f(self, *args, **kwargs)
  File "/opt/conda/default/lib/python3.6/site-packages/gcsfs/core.py", line 462, in _call
    validate_response(r, path)
  File "/opt/conda/default/lib/python3.6/site-packages/gcsfs/core.py", line 157, in validate_response
    raise FileNotFoundError
FileNotFoundError

I have invalided the cache and cleared instance cache before reading the file but they didn't help.

Versions:
gcsfs==0.3.0
dask==2.1.0

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the delete-and-reupload sequence described in the issue, then inspect gcsfs/core.py around _fetch_range and fsspec/spec.py around cache fetching. Done means reading the replacement object through file.read succeeds after the original object is deleted and reuploaded, with a regression test covering the sequence.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, python
Domain
cloud
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.