Generation ID ignored when reading from gcs
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 398
- Forks
- 181
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 36
Description
Hi, I am having an issue reading specific generation/versions of files from gcs.
For instance, in a bucket with versioning enabled:
# old
gs://mock-bucket/something.csv#123456789
# new
gs://mock-bucket/something.csv#999999999
when trying to read gs://mock-bucket/something.csv#123456789, it will read gs://mock-bucket/something.csv#999999999 without any warning or error message.
I am happy to make a contribution! let me know!
To reproduce:
- Create a gcs bucket with version enabled.
- Upload two file with different content to the bucket with the same key.
- This will generate 2 objects with same key but they will have different generation id
- use ffspec to read the object with older generation id
- ie:
from ffspec import open as f_open with f_open.read('gs://mock-bucket/something.csv#123456789', 'r') as f: .... - ffspec will only read ignore the generation id and only read the latest files.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the fsspec.open entry point for gs:// URLs and how the generation fragment is handled. Reproduce the case with a versioned Google Cloud Storage bucket; done means reading an older generation returns that content while requests without a generation still read the latest object.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, python
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100