[ECS] [request]: Publicly document the file system paths the SSM/ExecuteCommand agent writes to, so ECS Exec can be used with readonlyRootFilesystem: true
- Dominant language
- Shell
- Stars
- 5.4k
- Forks
- 334
- PR merge metrics
- No merged PRs in 30d
Description
## Community Note (per repo template)
> * Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
> * Please do not leave "+1" or other comments that do not add relevant new information or questions; they generate extra noise for issue followers and do not help prioritize the request.
> * If you are interested in working on this issue or have submitted a pull request, please leave a comment.
---
## Tell us about your request
Today, enabling [`readonlyRootFilesystem: true`](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_KernelCapabilities.html) on a container is **mutually exclusive** with [ECS Exec](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-exec.html). The official ECS Exec "Considerations" section states:
> The SSM agent requires that the container file system can be written to in order to create the required directories and files. Therefore, making the root file system read-only using the `readonlyRootFilesystem` task definition parameter, or any other method, isn't supported.
> — *[Amazon ECS Exec Considerations](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-exec.html#ecs-exec-considerations)*
However, **the exact set of paths the ECS managed agent and the embedded SSM/`amazon-ssm-agent` write to is not publicly documented anywhere**. Community experimentation (see [#1359](https://github.com/aws/containers-roadmap/issues/1359) and [amazon-ecs-exec-checker#21](https://github.com/aws-containers/amazon-ecs-exec-checker/issues/21)) suggests the following three paths, but this list is **neither officially confirmed nor version-pinned**:
| Path | Apparent purpose |
| --- | --- |
| `/managed-agents` | ECS managed-agent binary bind-mount target |
| `/var/lib/amazon/ssm` | SSM agent state / configuration |
| `/var/log/amazon/ssm` | SSM agent logs |
This issue asks for **two things, in priority order**:
1. **(Minimum ask — documentation)** Officially publish the **complete, versioned list of file system paths** that the ECS managed agent and the embedded SSM agent require write access to inside the application container, including any future additions. This alone unblocks customers, because we can then mount narrowly-scoped writable volumes (e.g. `tmpfs` or named volumes) at exactly those paths while keeping the rest of the root file system read-only.
2. **(Stretch ask — first-class support)** Officially support ECS Exec on tasks where `readonlyRootFilesystem: true`, by having the ECS agent automatically inject the required writable mounts (similar to how it already bind-mounts the managed agent binary today).
This is fundamentally a **specification transparency** request first, and a feature request second.
## Which service(s) is this request for?
Amazon ECS on Fargate and EC2 launch types.
## Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
We run ECS Fargate workloads under strict security baselines that mandate `readonlyRootFilesystem: true`. The requirement comes from multiple, independent sources:
- **CIS Docker Benchmark v1.6.0 — control 5.12**: "Ensure that the container's root filesystem is mounted as read only".
- **NIST SP 800-190 — Application Container Security Guide**: recommends immutable containers and read-only root file systems to limit blast radius of in-container compromise.
- **Internal / customer security audits** (SOC 2, ISO 27001, FedRAMP-aligned controls) frequently flag writable container root file systems as a finding.
- **Defense-in-depth against supply-chain and RCE exploits**: a read-only root file system meaningfully reduces what an attacker can persist after exploiting a vulnerable dependency.
At the same time, **ECS Exec is the AWS-recommended replacement for SSH-into-container** ([blog](https://aws.amazon.com/blogs/containers/new-using-amazon-ecs-exec-access-your-containers-fargate-ec2/)) for break-glass debugging, incident response, and on-call workflows — particularly on Fargate where there is no host to SSH into.
The current state forces customers to choose **exactly one** of:
- **(A) Secure baseline** — keep `readonlyRootFilesystem: true`, lose all ability to use ECS Exec for break-glass debugging on production Fargate tasks. Incident response degrades significantly.
- **(B) Operability** — drop `readonlyRootFilesystem`, fail security audits, weaken runtime hardening.
- **(C) Undocumented workaround** — mount writable volumes at the community-reverse-engineered paths above, and **hope** they remain stable across ECS agent and SSM agent upgrades. A silent change to required paths in a future agent release would break ECS Exec across our entire fleet with no warning, and we would have no contract to point to.
None of these are acceptable for regulated workloads.
The root cause of (C)'s fragility is exactly the missing public specification this issue is about.
## Are you currently working around this issue?
Yes — option (C) above. We mount the three community-known paths (`/managed-agents`, `/var/lib/amazon/ssm`, `/var/log/amazon/ssm`) as writable volumes while keeping `readonlyRootFilesystem: true`, and ECS Exec currently works. But:
- The path list is reverse-engineered from a community issue thread, not from AWS documentation.
- We have no SLA or deprecation policy on these paths.
- The [`amazon-ecs-exec-checker`](https://github.com/aws-containers/amazon-ecs-exec-checker) tool still flags our tasks as red because it only checks the `readonlyRootFilesystem` flag, not the mount layout.
- We cannot prove to auditors that this configuration is officially supported.
## Additional context
- Existing related issue: [#1359](https://github.com/aws/containers-roadmap/issues/1359) — open since 2021-04-30, currently labeled `Proposed`, with no public AWS response in 5+ years. Filing this as a separate, narrower issue (documentation of required paths) rather than commenting there, because the **documentation ask is materially smaller and independently actionable** than the full feature request.
- Related checker discussion: [aws-containers/amazon-ecs-exec-checker#21](https://github.com/aws-containers/amazon-ecs-exec-checker/issues/21).
- Comparable precedent: the AWS Fargate platform versions page documents the exact bind-mount layout for the task metadata endpoint and ephemeral storage. We are asking for the same level of disclosure for the ECS Exec / SSM agent.
- Our workload context:
## Attachments
N/A.
---
## Notes for the submitter (not part of the issue body)
- Before submitting, search the repo once more for the latest duplicate candidates: `is:issue readonlyRootFilesystem exec` and `is:issue "read-only" "ECS Exec"`.
- Consider also adding a 👍 reaction and a short comment on [#1359](https://github.com/aws/containers-roadmap/issues/1359) linking back to this new issue, to consolidate signal.
- If the AWS team prefers a single thread, be ready to close this and consolidate into #1359 — but the narrower documentation framing here is intentionally easier for them to action.
Contributor guide
Assessment
This issue has not been assessed yet.