aws / aws/aws-cli

s3 sync: Read include/exclude patterns from file

Open
#5,160 14 comments 13 reactions 0 assignees View on GitHub
feature-request p3 s3filters s3sync
Dominant language
Python
Stars
17.3k
Forks
4.6k
Avg merge
1d 2h
Merged PRs (30d)
13

Description

# Problem
When using `s3 sync`, we can include/exclude objects for the sync using `--exclude ` and `--include `. This can be cumbersome if someone has to deal with many patterns at once.

# Proposed solution
I'd suggest to implement two additional options: `--exclude-file ` and `--include-file ` so we would be able to specify files which contain lists of patterns. See how `rsync` does it [here](https://download.samba.org/pub/rsync/rsync.html). Implementing it in the same way as `rsync` does would have the benefit of maintaining one list for both.

Additional selling point: the exclude file can be managed in git without tampering with CI scripts.

## Example usage
### Now
```bash
aws s3 sync . s3://bucket/html/ --exclude=".*" --exclude "license/*" --exclude "LICENSE.md" --exclude "*.map" --exclude "*.scss" --exclude "customVariables.css" --exclude "css/bootstrap/*"
```

### With proposed feature
```bash
aws s3 sync . s3://bucket/html/ --exclude-file .syncignore
```

# Alternatives I have considered
None really. The only alternative I know of is using many single statements.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.