aws / aws/containers-roadmap

[Fargate] [request]: Mount S3 buckets as filesystem on Fargate, e.g. via FUSE

Open
#412 31 comments 160 reactions 1 assignee Claimed by @campmesh View on GitHub
Fargate Proposed
Dominant language
Shell
Stars
5.4k
Forks
334
PR merge metrics
No merged PRs in 30d

Description

**Tell us about your request**
A way of mounting (folders of) S3 buckets as a filesystem on Fargate containers, potentially via FUSE.

**Which service(s) is this request for?**
Fargate

**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**
We launch JupyterLab and RStudio servers on demand and per-user in Fargate containers, which are shut down when idle. Without file-persistance, source code and output data can be lost, and any previously uploaded data would have to be re-uploaded.

We also use AppStream for other analysis tools, which in our configuration stores the home folders on S3: so mounting these S3 folders on Fargate would be ideal to present a unified file-space for each user.

At the moment this isn't possible in Fargate since `--cap-add SYS_ADMIN` is not supported (and there may also be other requirements I'm not aware of).

While the below work-arounds are possible, they can be manual with a too high chance of mistakes.

**Are you currently working around this issue?**
Yes:
- For JupyterLab, we use an [S3 contents manager](https://github.com/uktrade/jupyters3) that can save and retrieve notebooks to/from S3 transparently.
- For other files / RStudio, users explicitly save/retrieve files from S3 using R or Python code, e.g. using Boto3 in Python

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.