aws / aws/aws-sam-cli

Feature request: local invocation should apply EphemeralStorage limits and report on usage

Open
#6,570 1 comment 0 reactions 0 assignees View on GitHub
area/local type/feature
Dominant language
Python
Stars
6.7k
Forks
1.2k
Avg merge
1d 10h
Merged PRs (30d)
52

Description

### Describe your idea/feature/enhancement

AWS Lambda functions have a [set limit for ephemeral storage](https://aws.amazon.com/blogs/compute/using-larger-ephemeral-storage-for-aws-lambda/) available at /tmp in the file system, defaulting to 512MB but able to be set up to 10GB as of 2022.

By default, Docker containers are allowed an [unlimited tmpfs](https://docs.docker.com/storage/tmpfs/#specify-tmpfs-options) which allows code to succeed in a local environment when it would inevitably fail in AWS, something you can only discover on deployment. Because of the nature of out-of-space errors in tmpfs (e.g `ENOSPC: no space left on device, mkdtemp '/tmp/...`), they can be caused by and present in varying/unreleated ways, depending on how an application or operating system handles the situation, making it hard to debug and identify the root cause.

By comparison, if SAM is able to set limits early, it will allow developers to identify issues locally and address them immediately, and make SAM/Docker more closely match AWS Lambda.

### Proposal

To assist with this, SAM can configure Docker with limits placed on tmpfs with the [`tmpfs-size` option](https://docs.docker.com/storage/tmpfs/#specify-tmpfs-options); this should be set at 512MB to match the Lambda defaults and be set based upon the [`EphemeralStorage` setting](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-lambda-function-ephemeralstorage.html) for the function in the template.

In a similar vein to #6260 for memory/RAM usage, SAM should report an error if a function invocation has reached its ephemeral storage limit. It would be useful to see a warning if an invocation is close to exceeding the limit also (within 20%, perhaps), but also an accounting of the tmpfs consumed, not just in error/warning situations. Ideally these messages would be coloured, like parts of SAM's build/invocation output.

### Additional Details

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.