custom-resource: Custom resource as a dependency for another custom resource
- Dominant language
- TypeScript
- Stars
- 12.9k
- Forks
- 4.6k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 83
Description
### Describe the bug
I am attempting to create a Neptune global database and then add a database cluster inside.
I am creating the global database using the `CustomResource` class, and the database cluster with the `AwsCustomResource` class.
The error that I am running into is that when I try to add a cluster to the global database, the deployment fails at creation as it cannot find the global database even after specifying that the cluster depends on the global database.
### Expected Behavior
I expect for the global databse to be fully created and available before the cluster is added.
### Current Behavior
11:43:46 PM | CREATE_FAILED | Custom::NeptuneRegionalCluster | NeptuneCluster7FC72740
Received response status [FAILED] from custom resource. Message returned: Global cluster global-database-identifier not found (RequestId: 83af0fa8-cea4-44d1-8ec7-9c2948986ce2)
❌ GlobalDB failed: Error: The stack named GlobalDB failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_FAILED (The following resource(s) failed to delete: [NeptuneCluster7FC72740]. ): Received response status [FAILED] from custom resource. Message returned: Global cluster global-cluster-identifier not found (RequestId: 83af0fa8-cea4-44d1-8ec7-9c2948986ce2), Received response status [FAILED] from custom resource. Message returned: Malformed db cluster arn dev-primary-cluster (RequestId: 2bfb458b-cfde-4169-b730-d8cfc0a258f7)
at FullCloudFormationDeployment.monitorDeployment (/usr/local/lib/node_modules/aws-cdk/lib/index.js:451:10568)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async Object.deployStack2 [as deployStack] (/usr/local/lib/node_modules/aws-cdk/lib/index.js:454:199716)
at async /usr/local/lib/node_modules/aws-cdk/lib/index.js:454:181438
❌ Deployment failed: Error: The stack named GlobalDB failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_FAILED (The following resource(s) failed to delete: [NeptuneCluster7FC72740]. ): Received response status [FAILED] from custom resource. Message returned: Global cluster global-cluster-identifier not found (RequestId: 83af0fa8-cea4-44d1-8ec7-9c2948986ce2), Received response status [FAILED] from custom resource. Message returned: Malformed db cluster arn dev-primary-cluster (RequestId: 2bfb458b-cfde-4169-b730-d8cfc0a258f7)
at FullCloudFormationDeployment.monitorDeployment (/usr/local/lib/node_modules/aws-cdk/lib/index.js:451:10568)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async Object.deployStack2 [as deployStack] (/usr/local/lib/node_modules/aws-cdk/lib/index.js:454:199716)
at async /usr/local/lib/node_modules/aws-cdk/lib/index.js:454:181438
### Reproduction Steps
`neptune.ts`
```typescript
import { Stack } from "aws-cdk-lib";
import { Code, CodeSigningConfig, Runtime } from "aws-cdk-lib/aws-lambda";
import { Platform, SigningProfile } from "aws-cdk-lib/aws-signer";
import { NodejsFunction } from "aws-cdk-lib/aws-lambda-nodejs";
class GlobalDatabaseStack extends Stack {
globalClusterIdentifier = "global-clulster-identifier"
engineVersion = "1.2.0.0"
constructor(scope: App, id: string) {
const signingProfile = new SigningProfile(this, "SigningProfile", {
platform: Platform.AWS_LAMBDA_SHA384_ECDSA,
});
const codeSigningConfig = new CodeSigningConfig(this, "CodeSigningConfig", {
signingProfiles: [signingProfile],
});
const globalClusterOnEventHandler = new NodejsFunction(
this,
"NeptuneGlobalClusterOnEventHandler",
{
codeSigningConfig,
runtime: Runtime.NODEJS_20_X,
handler: "globalClusterOnEventHandler/globalClusterOnEventHandler.handler",
code: Code.fromAsset(join(__dirname, "lambda", "globalClusterOnEventHandler", "globalClusterOnEventHandler.zip")),
bundling: {
externalModules: ["aws-sdk"],
},
}
);
const globalClusterProvider = new Provider(
this,
"NeptuneGlobalClusterProvider",
{
onEventHandler: globalClusterOnEventHandler,
}
);
// create global cluster
const globalCluster = new CustomResource(this, "NeptuneGlobalDatabase", {
serviceToken: globalClusterProvider.serviceToken,
properties: {
// stack: this,
GlobalClusterIdentifier: this.globalClusterIdentifier,
engineVersion: this.engineVersion,
},
resourceType: "Custom::NeptuneGlobalCluster",
});
// add a cluster in the primary rgion
const primaryCluster = new AwsCustomResource(this, "NeptuneCluster", {
onCreate: {
action: "CreateDBClusterCommand",
service: "@aws-sdk/client-neptune",
physicalResourceId: PhysicalResourceId.of(Date.now().toString()),
parameters: {
// required
DBClusterIdentifier: `dev-primary-cluster`,
Engine: "neptune",
DatabaseName: `globalDatabase`,
EngineVersion: this.engineVersion,
GlobalClusterIdentifier: this.globalClusterIdentifier, // db name
},
},
onDelete: {
action: "RemoveFromGlobalClusterCommand",
service: "@aws-sdk/client-neptune",
parameters: {
GlobalClusterIdentifier: this.globalClusterIdentifier,
DbClusterIdentifier: `dev-primary-cluster`,
},
},
resourceType: "Custom::NeptuneRegionalCluster",
policy: AwsCustomResourcePolicy.fromSdkCalls({
resources:AwsCustomResourcePolicy.ANY_RESOURCE
})
});
primaryCluster.node.addDependency(globalCluster);
}
```
`lambda/globalClusterOnEventHandler/globalClusterOnEventHandler.ts`
```typescript
import { AwsCustomResource, AwsCustomResourcePolicy, PhysicalResourceId } from "aws-cdk-lib/custom-resources";
import { CloudFormationCustomResourceEvent, Context } from "aws-lambda";
export const handler = async (
event: CloudFormationCustomResourceEvent,
context: Context
) => {
const { stack, GlobalClusterIdentifier, engineVersion, storageEncrypted } =
event.ResourceProperties;
let resp = {
LogicalResourceId: event.LogicalResourceId,
StackId: event.StackId,
RequestId: event.RequestId,
// PhysicalResourceId: context.functionName,
Status: "FAILED",
Reason: "",
Data: {},
};
switch (event.RequestType) {
case "Create":
let global;
try {
global = new AwsCustomResource(stack, "NeptuneGlobalDatabase", {
onCreate: {
action: "CreateGlobalClusterCommand",
service: "@aws-sdk/client-neptune",
physicalResourceId: PhysicalResourceId.of(Date.now().toString()),
parameters: {
// required
GlobalClusterIdentifier: GlobalClusterIdentifier, // db name
Engine: "neptune",
EngineVersion: engineVersion,
StorageEncrypted: storageEncrypted,
},
},
policy: AwsCustomResourcePolicy.fromSdkCalls({
resources: AwsCustomResourcePolicy.ANY_RESOURCE
})
});
} catch (error) {
resp.Status = "FAILED";
return resp;
}
resp.Status = "SUCCESS";
resp.Data = {
cluster: global
}
return resp;
case "Update":
case "Delete":
};
```
### Possible Solution
_No response_
### Additional Information/Context
_No response_
### CDK CLI Version
2.146.0 (build b368c78)
### Framework Version
_No response_
### Node.js Version
v20.3.0
### OS
Debian GNU/Linux 12 (bookworm) on Windows 10 x86_64 Home 22H2 | Kernel version: 5.15.153.1-microsoft-standard-WSL2
### Language
TypeScript
### Language Version
_No response_
### Other information
_No response_
Contributor guide
Research direction
Start with the dependency setup in neptune.ts and then inspect lambda/globalClusterOnEventHandler/globalClusterOnEventHandler.ts, especially how the custom resource reports completion. Reproduce the deployment with the shown Neptune global and regional cluster definitions; done means the regional cluster no longer fails because the global cluster is unavailable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, node.js, typescript
- Domain
- cloud, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100