GoogleCloudPlatform / GoogleCloudPlatform/gsutil

gsutil rsync and dot-named directory in a bucket

Open
#1,130 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
918
Forks
335
PR merge metrics
No merged PRs in 30d

Description

We sync local directory with a bucket using gsutil rsync:
gsutil -m rsync -d -r "gs://${GCS_BUCKET}/plugins" "${base_dir}/plugins"

Recently we had a problem - a file was updated in the bucket but in the local directory we still saw an old version.
We discovered that it was because of a dot-named directory in bucket.
So we had files like:
`plugins/plugins/myfile.py`
`plugins/./plugins/myfile.py`
and after they are copied to linux system they actually point to the same path (so one overrode each other)

Is there anything that could be done? Eg warn a user, or even escape this directory (so both files are copied).

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the reported `gsutil -m rsync -d -r` command with bucket objects containing both `plugins/plugins/myfile.py` and `plugins/./plugins/myfile.py`. Trace how rsync maps object names to local paths, then define and test whether the finished behavior should warn about collisions or preserve both objects without overwriting.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud
Domain
cli, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.