aws / aws/aws-cdk

efs - ecs : Cannot re-mount an existing efs

Open
#26,537 16 comments 0 reactions 0 assignees View on GitHub
@aws-cdk/aws-efs bug effort/medium p2
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
2d 3h
Merged PRs (30d)
83

Description

### Describe the bug

Hi, hope to find you well. I am trying to mount an existing EFS to a redis ECS Task. Everything works smoothly the first creation, but no luck when trying to remount the same FS which returns a puzzling error.

### Expected Behavior

I should be able to remount the EFS, afterall what is the point of the RetainPolicy otherwise?

### Current Behavior

This is my code:

```ts
const qmmTasksEfsSecurityGroup = new ec2.SecurityGroup(this, `${props.STAGE}qmmTasksEfsSecurityGroup`, {
vpc: props.vpc,
securityGroupName: `${props.STAGE}qmmTasksEfsSecurityGroup`
})

let qmmTasksEfs: efs.IFileSystem
let qmmRedisEfsAccessPoint: efs.IAccessPoint
let qmmMongoEfsAccessPoint: efs.IAccessPoint

// if (true) {
if (props.recreated) {

qmmTasksEfs = new efs.FileSystem(this, `${props.STAGE}qmmTasksEfs`, {
fileSystemName: `${props.STAGE}qmmTasksEfs`,
vpc: props.vpc,
removalPolicy: cdk.RemovalPolicy.RETAIN,
securityGroup: qmmTasksEfsSecurityGroup,
encrypted: true,
lifecyclePolicy: efs.LifecyclePolicy.AFTER_30_DAYS,
enableAutomaticBackups: true
})

new cdk.CfnOutput(this, 'QlashMainClusterEFSID', {
exportName: 'QlashMainClusterEFSID',
value: qmmTasksEfs.fileSystemId
})

qmmRedisEfsAccessPoint = new efs.AccessPoint(this, `${props.STAGE}qmmRedisAccessPoint`, {
fileSystem: qmmTasksEfs,
path: '/redis',
createAcl: {
ownerGid: '1001',
ownerUid: '1001',
permissions: '750'
},
posixUser: {
uid: '1001',
gid: '1001'
}
})

qmmRedisEfsAccessPoint.applyRemovalPolicy(cdk.RemovalPolicy.RETAIN)

new cdk.CfnOutput(this, 'QlashMainClusterRedisAccessPointID', {
exportName: 'QlashMainClusterRedisAccessPointID',
value: qmmRedisEfsAccessPoint.accessPointId
})

qmmMongoEfsAccessPoint = new efs.AccessPoint(this, `${props.STAGE}qmmMongoAccessPoint`, {
fileSystem: qmmTasksEfs,
path: '/mongodb',
createAcl: {
ownerGid: '1002',
ownerUid: '1002',
permissions: '750'
},
posixUser: {
uid: '1002',
gid: '1002'
}
})

qmmMongoEfsAccessPoint.applyRemovalPolicy(cdk.RemovalPolicy.RETAIN)

new cdk.CfnOutput(this, 'QlashMainClusterMongoAccessPointID', {
exportName: 'QlashMainClusterMongoAccessPointID',
value: qmmMongoEfsAccessPoint.accessPointId
})

} else {

qmmTasksEfs = efs.FileSystem.fromFileSystemAttributes(this, `${props.STAGE}qmmTasksEfs`, {
securityGroup: qmmTasksEfsSecurityGroup,
fileSystemId: config.QlashMainClusterEFSID
})

qmmRedisEfsAccessPoint = efs.AccessPoint.fromAccessPointId(this, `${props.STAGE}qmmRedisAccessPoint`, config.QlashMainClusterRedisAccessPointID)

qmmMongoEfsAccessPoint = efs.AccessPoint.fromAccessPointId(this, `${props.STAGE}qmmMongoAccessPoint`, config.QlashMainClusterMongoAccessPointID)
}

// Redis

const qmmRedisServiceSecurityGroup = new ec2.SecurityGroup(this, `${props.STAGE}qmmRedisSecurityGroup`, {
vpc: props.vpc,
securityGroupName: `${props.STAGE}qmmRedisSecurityGroup`
})

qmmTasksEfsSecurityGroup.addIngressRule(
ec2.Peer.securityGroupId(qmmRedisServiceSecurityGroup.securityGroupId),
ec2.Port.tcp(2049),
'Allow inbound traffic from qmm_redis to qmmTasksEfs'
)

if (props.qlashMainInstanceSecurityGroup) {
qmmRedisServiceSecurityGroup.addIngressRule(
ec2.Peer.securityGroupId(props.qlashMainInstanceSecurityGroup.securityGroupId),
ec2.Port.tcp(6379),
'Allow inbound traffic to qmm_redis from qmmMain instance'
)
}

qmmRedisServiceSecurityGroup.addIngressRule(
ec2.Peer.ipv4(props.vpc.vpcCidrBlock),
ec2.Port.tcp(6379),
'Allow inbound traffic to qmm_redis from resources in qlashMainClusterVpc'
)

const qmmRedisTaskDefinition = new ecs.FargateTaskDefinition(this, `${props.STAGE.toLowerCase()}qmmRedisTask`, {
cpu: 2048,
memoryLimitMiB: 8192,
volumes: [
{
name: `${props.STAGE.toLowerCase()}_qmm_redis_volume`,
efsVolumeConfiguration: {
fileSystemId: qmmTasksEfs.fileSystemId,
transitEncryption: 'ENABLED',
authorizationConfig: {
accessPointId: qmmRedisEfsAccessPoint.accessPointId,
iam: 'ENABLED'
}
}
}
]
})

qmmRedisTaskDefinition.addToTaskRolePolicy(
new iam.PolicyStatement({
actions: [
'elasticfilesystem:ClientWrite',
'elasticfilesystem:ClientMount',
'elasticfilesystem:ClientRootAccess',
'elasticfilesystem:DescribeMountTargets',
'elasticfilesystem:CreateAccessPoint',
'elasticfilesystem:DeleteAccessPoint'
],
resources: [qmmTasksEfs.fileSystemArn],
})
)

qmmRedisTaskDefinition.addToTaskRolePolicy(
new iam.PolicyStatement({
actions: [
'elasticfilesystem:DescribeAccessPoints',
'elasticfilesystem:DescribeFileSystems'
],
resources: ["*"],
})
)

qmmRedisTaskDefinition.addToTaskRolePolicy(
new iam.PolicyStatement({
actions: ['ec2:DescribeAvailabilityZones'],
resources: ['*']
})
)

```

### Reproduction Steps

When running the following code trying to remount the EFS, you will get this error:
```
ResourceInitializationError: failed to invoke EFS utils commands to set up EFS volumes: stderr: Failed to resolve "fs-006afd6cee7891114.efs.eu-central-1.amazonaws.com" - check that your file system ID is correct, and ensure that the VPC has an EFS mount target for this file system ID. See https://docs.aws.amazon.com/console/efs/mount-dns-name for more detail. Attempting to lookup mount target ip address using botocore. Failed to import necessary dependency botocore, please install botocore first. : unsuccessful EFS utils command execution; code: 1
```

What realy sounds strange is this:
> Attempting to lookup mount target ip address using botocore. Failed to import necessary dependency botocore, please install botocore first

### Possible Solution

I don't even know if this is something that is up to you guys or if it is an internal error from EFS itself

### Additional Information/Context

I've tried giving the task permissions on everything, just to check if it was a permission issue, but to no good

### CDK CLI Version

2.88

### Framework Version

_No response_

### Node.js Version

v18.15.0

### OS

Linux - Ubuntu

### Language

Typescript

### Language Version

_No response_

### Other information

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with the ECS task definition's EFS volume configuration shown in the issue and compare the synthesized configuration for a new versus imported file system. Reproduce the mount failure while checking the EFS mount target and DNS requirements; done means determining whether AWS CDK emits an incorrect configuration or confirming that the failure is an AWS EFS or ECS issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, typescript
Domain
cloud, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.