efs - ecs : Cannot re-mount an existing efs
- Dominant language
- TypeScript
- Stars
- 12.9k
- Forks
- 4.6k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 83
Description
### Describe the bug
Hi, hope to find you well. I am trying to mount an existing EFS to a redis ECS Task. Everything works smoothly the first creation, but no luck when trying to remount the same FS which returns a puzzling error.
### Expected Behavior
I should be able to remount the EFS, afterall what is the point of the RetainPolicy otherwise?
### Current Behavior
This is my code:
```ts
const qmmTasksEfsSecurityGroup = new ec2.SecurityGroup(this, `${props.STAGE}qmmTasksEfsSecurityGroup`, {
vpc: props.vpc,
securityGroupName: `${props.STAGE}qmmTasksEfsSecurityGroup`
})
let qmmTasksEfs: efs.IFileSystem
let qmmRedisEfsAccessPoint: efs.IAccessPoint
let qmmMongoEfsAccessPoint: efs.IAccessPoint
// if (true) {
if (props.recreated) {
qmmTasksEfs = new efs.FileSystem(this, `${props.STAGE}qmmTasksEfs`, {
fileSystemName: `${props.STAGE}qmmTasksEfs`,
vpc: props.vpc,
removalPolicy: cdk.RemovalPolicy.RETAIN,
securityGroup: qmmTasksEfsSecurityGroup,
encrypted: true,
lifecyclePolicy: efs.LifecyclePolicy.AFTER_30_DAYS,
enableAutomaticBackups: true
})
new cdk.CfnOutput(this, 'QlashMainClusterEFSID', {
exportName: 'QlashMainClusterEFSID',
value: qmmTasksEfs.fileSystemId
})
qmmRedisEfsAccessPoint = new efs.AccessPoint(this, `${props.STAGE}qmmRedisAccessPoint`, {
fileSystem: qmmTasksEfs,
path: '/redis',
createAcl: {
ownerGid: '1001',
ownerUid: '1001',
permissions: '750'
},
posixUser: {
uid: '1001',
gid: '1001'
}
})
qmmRedisEfsAccessPoint.applyRemovalPolicy(cdk.RemovalPolicy.RETAIN)
new cdk.CfnOutput(this, 'QlashMainClusterRedisAccessPointID', {
exportName: 'QlashMainClusterRedisAccessPointID',
value: qmmRedisEfsAccessPoint.accessPointId
})
qmmMongoEfsAccessPoint = new efs.AccessPoint(this, `${props.STAGE}qmmMongoAccessPoint`, {
fileSystem: qmmTasksEfs,
path: '/mongodb',
createAcl: {
ownerGid: '1002',
ownerUid: '1002',
permissions: '750'
},
posixUser: {
uid: '1002',
gid: '1002'
}
})
qmmMongoEfsAccessPoint.applyRemovalPolicy(cdk.RemovalPolicy.RETAIN)
new cdk.CfnOutput(this, 'QlashMainClusterMongoAccessPointID', {
exportName: 'QlashMainClusterMongoAccessPointID',
value: qmmMongoEfsAccessPoint.accessPointId
})
} else {
qmmTasksEfs = efs.FileSystem.fromFileSystemAttributes(this, `${props.STAGE}qmmTasksEfs`, {
securityGroup: qmmTasksEfsSecurityGroup,
fileSystemId: config.QlashMainClusterEFSID
})
qmmRedisEfsAccessPoint = efs.AccessPoint.fromAccessPointId(this, `${props.STAGE}qmmRedisAccessPoint`, config.QlashMainClusterRedisAccessPointID)
qmmMongoEfsAccessPoint = efs.AccessPoint.fromAccessPointId(this, `${props.STAGE}qmmMongoAccessPoint`, config.QlashMainClusterMongoAccessPointID)
}
// Redis
const qmmRedisServiceSecurityGroup = new ec2.SecurityGroup(this, `${props.STAGE}qmmRedisSecurityGroup`, {
vpc: props.vpc,
securityGroupName: `${props.STAGE}qmmRedisSecurityGroup`
})
qmmTasksEfsSecurityGroup.addIngressRule(
ec2.Peer.securityGroupId(qmmRedisServiceSecurityGroup.securityGroupId),
ec2.Port.tcp(2049),
'Allow inbound traffic from qmm_redis to qmmTasksEfs'
)
if (props.qlashMainInstanceSecurityGroup) {
qmmRedisServiceSecurityGroup.addIngressRule(
ec2.Peer.securityGroupId(props.qlashMainInstanceSecurityGroup.securityGroupId),
ec2.Port.tcp(6379),
'Allow inbound traffic to qmm_redis from qmmMain instance'
)
}
qmmRedisServiceSecurityGroup.addIngressRule(
ec2.Peer.ipv4(props.vpc.vpcCidrBlock),
ec2.Port.tcp(6379),
'Allow inbound traffic to qmm_redis from resources in qlashMainClusterVpc'
)
const qmmRedisTaskDefinition = new ecs.FargateTaskDefinition(this, `${props.STAGE.toLowerCase()}qmmRedisTask`, {
cpu: 2048,
memoryLimitMiB: 8192,
volumes: [
{
name: `${props.STAGE.toLowerCase()}_qmm_redis_volume`,
efsVolumeConfiguration: {
fileSystemId: qmmTasksEfs.fileSystemId,
transitEncryption: 'ENABLED',
authorizationConfig: {
accessPointId: qmmRedisEfsAccessPoint.accessPointId,
iam: 'ENABLED'
}
}
}
]
})
qmmRedisTaskDefinition.addToTaskRolePolicy(
new iam.PolicyStatement({
actions: [
'elasticfilesystem:ClientWrite',
'elasticfilesystem:ClientMount',
'elasticfilesystem:ClientRootAccess',
'elasticfilesystem:DescribeMountTargets',
'elasticfilesystem:CreateAccessPoint',
'elasticfilesystem:DeleteAccessPoint'
],
resources: [qmmTasksEfs.fileSystemArn],
})
)
qmmRedisTaskDefinition.addToTaskRolePolicy(
new iam.PolicyStatement({
actions: [
'elasticfilesystem:DescribeAccessPoints',
'elasticfilesystem:DescribeFileSystems'
],
resources: ["*"],
})
)
qmmRedisTaskDefinition.addToTaskRolePolicy(
new iam.PolicyStatement({
actions: ['ec2:DescribeAvailabilityZones'],
resources: ['*']
})
)
```
### Reproduction Steps
When running the following code trying to remount the EFS, you will get this error:
```
ResourceInitializationError: failed to invoke EFS utils commands to set up EFS volumes: stderr: Failed to resolve "fs-006afd6cee7891114.efs.eu-central-1.amazonaws.com" - check that your file system ID is correct, and ensure that the VPC has an EFS mount target for this file system ID. See https://docs.aws.amazon.com/console/efs/mount-dns-name for more detail. Attempting to lookup mount target ip address using botocore. Failed to import necessary dependency botocore, please install botocore first. : unsuccessful EFS utils command execution; code: 1
```
What realy sounds strange is this:
> Attempting to lookup mount target ip address using botocore. Failed to import necessary dependency botocore, please install botocore first
### Possible Solution
I don't even know if this is something that is up to you guys or if it is an internal error from EFS itself
### Additional Information/Context
I've tried giving the task permissions on everything, just to check if it was a permission issue, but to no good
### CDK CLI Version
2.88
### Framework Version
_No response_
### Node.js Version
v18.15.0
### OS
Linux - Ubuntu
### Language
Typescript
### Language Version
_No response_
### Other information
_No response_
Contributor guide
Research direction
Start with the ECS task definition's EFS volume configuration shown in the issue and compare the synthesized configuration for a new versus imported file system. Reproduce the mount failure while checking the EFS mount target and DNS requirements; done means determining whether AWS CDK emits an incorrect configuration or confirming that the failure is an AWS EFS or ECS issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, typescript
- Domain
- cloud, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100