s3 sync does not delete excluded files
- Dominant language
- Python
- Stars
- 17.3k
- Forks
- 4.6k
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 13
Description
the `--delete` flag on the s3 sync command does exactly what it says in the manual:
> Files that exist in the destination but not in the source are deleted during sync.
Meaning that it will not delete files in the destination if they exist in the source *even if they were excluded*. To reproduce:
```bash
aws s3 mb s3://sync-delete-exclude
mkdir /tmp/sync-delete-exclude
touch /tmp/sync-delete-exclude/{1,2,3}
aws s3 sync /tmp/sync-delete-exclude/ s3://sync-delete-exclude
# we expect this to delete the file 3
aws s3 sync s3://sync-delete-exclude --exclude=3 --delete
```
Our use case is to sync only the last week/month/year of files out of a s3 bucket using exclude and include filters, but files that are excluded are not deleted, meaning we must invoke another step afterwards to delete excluded files. This seems far off from the intent of the command which is "make the destination look like the source after filtering"
Contributor guide
Assessment
This issue has not been assessed yet.