aws / aws/aws-sam-cli

Deduplicate S3 Artifact Uploads in sam package/deploy

Open
#3,301 5 comments 0 reactions 0 assignees View on GitHub
area/deploy area/performance type/feature
Dominant language
Python
Stars
6.7k
Forks
1.2k
Avg merge
1d 10h
Merged PRs (30d)
52

Description

### Describe your idea/feature/enhancement

When I run `sam deploy` with many functions sharing the same `CodeUri`, SAM CLI tries to upload the same artifacts many times:

```
Uploading to xxx/ed720a86f3c46571e8dfb831d6axxx 10893967 / 10893967 (100.00%)
File with same data already exists at xxx/ed720a86f3c46571e8dfb831d6axxx, skipping upload
File with same data already exists at xxx/ed720a86f3c46571e8dfb831d6axxx, skipping upload
File with same data already exists at xxx/ed720a86f3c46571e8dfb831d6axxx, skipping upload
File with same data already exists at xxx/ed720a86f3c46571e8dfb831d6axxx, skipping upload
File with same data already exists at xxx/ed720a86f3c46571e8dfb831d6axxx, skipping upload
File with same data already exists at xxx/ed720a86f3c46571e8dfb831d6axxx, skipping upload
File with same data already exists at xxx/ed720a86f3c46571e8dfb831d6axxx, skipping upload
File with same data already exists at xxx/ed720a86f3c46571e8dfb831d6axxx, skipping upload
<... many times>
File with same data already exists at xxx/ed720a86f3c46571e8dfb831d6axxx, skipping upload
File with same data already exists at xxx/ed720a86f3c46571e8dfb831d6axxx, skipping upload
```

Each check/upload takes a few secs and they add up quickly when I have many functions.

### Proposal

Add details of how to add this to the product.

Things to consider:
1. Optimize the upload process so that the artifacts with the same hash (or object key) is only checked and uploaded once.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.