aws / aws/containers-roadmap

[service] [request]: Respect empty string variables set via `environmentFiles` in Fargate

Open
#2,466 0 comments 1 reaction 0 assignees View on GitHub
ECS Fargate Proposed
Dominant language
Shell
Stars
5.4k
Forks
334
PR merge metrics
No merged PRs in 30d

Description

> [!NOTE]
>
> This GitHub issue corresponds to AWS Support Case 172722539600462. AWS Support advised me to open this GitHub issue for public visibility and tracking.

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment

### Tell us about your request

ECS Fargate support for `.env` files subtly differs from that of Docker. Specifically, empty environment variable entries (e.g., `EMPTY=`) are propagated via `docker run --env-file`, are ignored within Fargate ECS. ECS documentation states that `environmentFiles` behaves like `docker run --env-file`, so this request is to alleviate this discrepancy within Fargate ECS.

### Which service(s) is this request for?

This is observed on Fargate ECS. (It possible also affects Fargate EKS, I did not test it.)

### Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?

[ECS documentation on `environmentFiles`](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/use-environment-file.html) seems to convey that the behavior of `environmentFiles` on Fargate matches that of Docker:

> The following rules apply to the Fargate launch type:
>
> The file is handled similar to a native Docker env-file.

However, the way ECS Fargate's handling of `environmentFiles` differs from Docker's handling of `--env-file` (see reproduction steps below).

### Are you currently working around this issue?

This problem affects `environmentFiles`, but not `environment`. It is possible to pass an empty environment variable via `environment`:

```json
"environment": [
{
"name": "VAR_EMPTY",
"value": ""
}
]
```

### Additional context

Step-by-step reproduction

#### Docker behavior

1. Create the following file (`vars.env`) which defines two environment variables:

```
VAR_NONEMPTY=value
VAR_EMPTY=
```

2. Verify that `docker run` propagates _both_ variables to the container:

```
$ docker run --rm --env-file vars.env alpine sh -c 'env | grep ^VAR_'
VAR_NONEMPTY=value
VAR_EMPTY=
```

#### ECS behavior

1. Upload `vars.env` to S3 (`s3://ecs-env-file-test/vars.env`).

2. Create a new IAM role to use as an ECS task execution role (`EcsEnvFileTestExecutionRole`).

3. Allow ECS to assume `EcsEnvFileTestExecutionRole` via the following trust policy:

```json
{
"Version": "2012-10-17",
"Statement": {
"Effect": "Allow",
"Principal": {
"Service": "ecs-tasks.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
}
```

4. Allow `EcsEnvFileTestExecutionRole` to perform basic ECS execution role tasks by adding the AWS-managed policy `AmazonECSTaskExecutionRolePolicy` to its permissions.

5. Allow `EcsEnvFileTestExecutionRole` to read `s3://ecs-env-file-test/vars.env` by adding the following inline policy to its permissions:

```json
{
"Version": "2012-10-17",
"Statement": {
"Effect": "Allow",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::ecs-env-file-test/vars.env"
}
}
```

6. Create the default ECS cluster.

7. Create a CloudWatch Logs group (`/ecs/EnvFileTest`) for ECS to write logs to.

8. Create ECS task definition (`EnvFileTest`) with JSON:

```json
{
"requiresCompatibilities": [
"FARGATE"
],
"family": "EnvFileTest",
"containerDefinitions": [
{
"name": "alpine",
"image": "alpine",
"command": [
"sh",
"-c",
"env | grep ^VAR_"
],
"environmentFiles": [
{
"value": "arn:aws:s3:::ecs-env-file-test/vars.env",
"type": "s3"
}
],
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-create-group": "true",
"awslogs-group": "/ecs/EnvFileTest",
"awslogs-region": "us-west-2",
"awslogs-stream-prefix": "ecs"
}
}
}
],
"networkMode": "awsvpc",
"memory": "512",
"cpu": "256",
"executionRoleArn": "arn:aws:iam::XXXXXXXXXXXX:role/EcsEnvFileTestExecutionRole"
}
```

9. Run task definition `EnvFileTest` on Fargate on the default ECS cluster. For task networking options, choose the default VPC and assign the default security group. Keep the "Public IP" toggle on to allow ECS to pull images from the Docker Hub registry and fetch the environment file from S3 (which is simpler than the alternative option of setting up VPC endpoints).

10. Examine the CloudWatch logs from the container, and notice that **the environment variable `VAR_EMPTY` is missing**:

```
VAR_NONEMPTY=value
```

This differs from the behavior of `docker run --env-file`.

11. (Side point) If the same ECS task is run on EC2 instead of Fargate, `VAR_EMPTY` is present (which matches `docker run --env-file`). I did not try ECS Anywhere (`EXTERNAL` deployment) option. AWS Support kindly pointed out that this is because of acommunity contribution: https://github.com/aws/amazon-ecs-agent/pull/3797

Contributor guide

Open the contributing guide

Research direction

Start with the documented reproduction using Docker's --env-file and an ECS Fargate task definition with environmentFiles, then compare the behavior with the ECS-on-EC2 result. Review the referenced amazon-ecs-agent pull request 3797 for related context. Done means Fargate preserves variables declared with empty values in environment files, matching Docker behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, docker
Domain
cloud, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.