Pin generation on open for version aware file system
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 398
- Forks
- 181
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 36
Description
Here is an example of the current behavior of version_aware file systems and open files in GCSFS.
I would have expected a version_aware file system to pin the generation of an object while the file is open so that the reads are consistent like they are when the open url specifies the generation.
If this is the desired behavior, I would be happy to take a look and see if I can fix it. Based on your comment in the previous issue I think the python gcs sdk download_as_bytes api should support what we need implementing etag matching similar to what exists in S3 but I haven't compared the details yet.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the behavior from the linked example with a version-aware GCSFS filesystem and an open file. Read the version-aware open/read path and compare it with the existing S3 etag handling, then check whether Google Cloud Storage's download_as_bytes API supports generation or etag matching; done means reads remain consistent while the file is open.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, python
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100