Add rsync-like --update to s3 sync.
- Dominant language
- Python
- Stars
- 17.3k
- Forks
- 4.6k
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 13
Description
It would be very powerful if sync copied the behavior of [rsync](http://www.samba.org/ftp/rsync/rsync.html)'s --update flag. This would allows predictable syncing from multiple directories without clobbering newer files. As it behaves right now, sync will update a file if the size OR timestamp differs.
In this example I'm creating a two folders, each with one file (test_file) having different sizes. I then create a bucket, sync the older, then the newer, then the older. As you can see from the output, the newer file is overwritten.
```
mkdir srcA srcB;
echo "older" > srcA/test_file;
sleep 2;
echo "this is newer" > srcB/test_file;
alias aws='aws --region=us-east-1';
ACL="--acl public-read";
bucket=$(uuidgen);
aws s3api create-bucket $ACL --bucket $bucket
cd srcA; aws s3 sync $ACL . s3://$bucket/test/; cd ..
curl http://$bucket.s3.amazonaws.com/test/test_file
cd srcB; aws s3 sync $ACL . s3://$bucket/test/; cd ..
curl http://$bucket.s3.amazonaws.com/test/test_file
cd srcA; aws s3 sync $ACL . s3://$bucket/test/; cd ..
curl http://$bucket.s3.amazonaws.com/test/test_file
aws s3 rm s3://$bucket/test/test_file
aws s3api delete-bucket --bucket $bucket
```
```
=> upload: test_file to s3://a0dab2e5-2c66-49d8-971f-1bee05299ff3/test/test_file
=> older
=> upload: test_file to s3://a0dab2e5-2c66-49d8-971f-1bee05299ff3/test/test_file
=> this is newer
=> upload: test_file to s3://a0dab2e5-2c66-49d8-971f-1bee05299ff3/test/test_file
=> older
```
With the --update flag in place like this:
```
cd srcA; aws s3 sync $ACL --update . s3://$bucket/test/; cd ..
cd srcB; aws s3 sync $ACL --update . s3://$bucket/test/; cd ..
cd srcA; aws s3 sync $ACL --update . s3://$bucket/test/; cd ..
curl http://$bucket.s3.amazonaws.com/test/test_file
```
I would expect the following output:
```
=> upload: test_file to s3://a0dab2e5-2c66-49d8-971f-1bee05299ff3/test/test_file
=> upload: test_file to s3://a0dab2e5-2c66-49d8-971f-1bee05299ff3/test/test_file
# no output from the third command
=> this is newer
```
Contributor guide
Assessment
This issue has not been assessed yet.