GoogleCloudPlatform / GoogleCloudPlatform/gsutil
Rsync Race Condition
- Dominant language
- Python
- Stars
- 918
- Forks
- 335
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
I think I'm experiencing a race condition when using rsync between a bucket and local disk (also possibly between two buckets).
If a file is deleted from the bucket after the _DiffIterator object is created I am seeing the following:
`NotFoundException: 404 gs://xxxxxxxxxxxxxxxxx`
This bucket is pretty busy with read/write operations so it is likely a file can be deleted after the diff is generated.
Is this expected?
This is problematic as the script exits with a non-zero code and I'm trying to implement some functionality using Kubernetes CronJobs.
I'm using the following container: **google/cloud-sdk:351.0.0-slim**
Thanks
Contributor guide
Research direction
Start with the gsutil rsync path around the mentioned _DiffIterator and reproduce the deletion-after-diff case using google/cloud-sdk:351.0.0-slim. Determine the expected behavior when a bucket object disappears and verify the resulting exit status for a Kubernetes CronJob workflow.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, kubernetes, python
- Domain
- cli, cloud, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100