aws / aws/aws-cli

s3 sync does not delete excluded files

Open
#4,923 7 comments 9 reactions 0 assignees View on GitHub
feature-request p3 s3filters s3sync
Dominant language
Python
Stars
17.3k
Forks
4.6k
Avg merge
1d 2h
Merged PRs (30d)
13

Description

the `--delete` flag on the s3 sync command does exactly what it says in the manual:

> Files that exist in the destination but not in the source are deleted during sync.

Meaning that it will not delete files in the destination if they exist in the source *even if they were excluded*. To reproduce:

```bash
aws s3 mb s3://sync-delete-exclude
mkdir /tmp/sync-delete-exclude
touch /tmp/sync-delete-exclude/{1,2,3}
aws s3 sync /tmp/sync-delete-exclude/ s3://sync-delete-exclude
# we expect this to delete the file 3
aws s3 sync s3://sync-delete-exclude --exclude=3 --delete
```

Our use case is to sync only the last week/month/year of files out of a s3 bucket using exclude and include filters, but files that are excluded are not deleted, meaning we must invoke another step afterwards to delete excluded files. This seems far off from the intent of the command which is "make the destination look like the source after filtering"

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.