aws / aws/aws-cli

Feature: Add option for "sync" command to use S3 Inventory manifest.json file

Open
#2,463 4 comments 0 reactions 0 assignees View on GitHub
automation-exempt feature-request p3 s3sync
Dominant language
Python
Stars
17.3k
Forks
4.6k
Avg merge
1d 2h
Merged PRs (30d)
13

Description

For the "sync" command operating with a local source path to a S3 bucket, such as "aws sync . s3://example-bucket", please add the ability to specify a path to an S3 Inventory manifest file.

For example: aws sync --inventory-manifest=s3://example-bucket/destination-prefix/source-bucket/config-ID/YYYY-MM-DDTHH-MMZ/manifest.json . s3://example-bucket

With the option present, the sync command will retrieve the Inventory list, uncompress, compare the local filesystem's files to the Inventory file, and sync the files as normal to the S3 bucket.

This benefit of this is the sync command will avoid having to run synchronous S3 API List operations for the comparisons and should result in significant sync speed up and cost reductions.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.