Deduplicate S3 Artifact Uploads in sam package/deploy
- Dominant language
- Python
- Stars
- 6.7k
- Forks
- 1.2k
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 52
Description
### Describe your idea/feature/enhancement
When I run `sam deploy` with many functions sharing the same `CodeUri`, SAM CLI tries to upload the same artifacts many times:
```
Uploading to xxx/ed720a86f3c46571e8dfb831d6axxx 10893967 / 10893967 (100.00%)
File with same data already exists at xxx/ed720a86f3c46571e8dfb831d6axxx, skipping upload
File with same data already exists at xxx/ed720a86f3c46571e8dfb831d6axxx, skipping upload
File with same data already exists at xxx/ed720a86f3c46571e8dfb831d6axxx, skipping upload
File with same data already exists at xxx/ed720a86f3c46571e8dfb831d6axxx, skipping upload
File with same data already exists at xxx/ed720a86f3c46571e8dfb831d6axxx, skipping upload
File with same data already exists at xxx/ed720a86f3c46571e8dfb831d6axxx, skipping upload
File with same data already exists at xxx/ed720a86f3c46571e8dfb831d6axxx, skipping upload
File with same data already exists at xxx/ed720a86f3c46571e8dfb831d6axxx, skipping upload
<... many times>
File with same data already exists at xxx/ed720a86f3c46571e8dfb831d6axxx, skipping upload
File with same data already exists at xxx/ed720a86f3c46571e8dfb831d6axxx, skipping upload
```
Each check/upload takes a few secs and they add up quickly when I have many functions.
### Proposal
Add details of how to add this to the product.
Things to consider:
1. Optimize the upload process so that the artifacts with the same hash (or object key) is only checked and uploaded once.
Contributor guide
Assessment
This issue has not been assessed yet.