(lambda-event-sources): Unable to specify dynamic topic name in kafka
- Dominant language
- TypeScript
- Stars
- 12.9k
- Forks
- 4.6k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 83
Description
### Describe the bug
A Kafka `kafkaTopic` with a dynamic name cannot be specified within a `ManagedKafkaEventSource`. An error is thrown when an unresolved token is provided.
This is because the ID for the event source mapping created during `bind` contains the topic name which means during synthesis, when the logical ID for the event source mapping is generated, a token is present in the ID for that construct which throws an error within the `uniqueid` [module](https://github.com/aws/aws-cdk/blob/7e424fdc45f98c83f0b19846c9683d7eeb767d80/packages/%40aws-cdk/core/lib/private/uniqueid.ts#L42)
### Expected Behavior
Dynamic topic name can be used and no error should be thrown during sythesis
### Current Behavior
Error thrown during synthesis
```
Error: Resolution error: ID components may not include unresolved tokens: KafkaEventSource:PartnerIntegrationsStacklocalGuidewireGuidewireSubroReportSenderEFF9AC27${Token[TOKEN.304]}.
Object creation stack:
at Function.uncachedString (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/lazy.ts:187:27)
at new CfnElement (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/cfn-element.ts:59:27)
at new CfnRefElement (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/cfn-element.ts:146:1)
at new CfnResource (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/cfn-resource.ts:85:5)
at new CfnEventSourceMapping (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/aws-lambda/lib/lambda.generated.ts:1778:9)
at new EventSourceMapping (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/aws-lambda/lib/event-source-mapping.ts:316:35)
at Function.addEventSourceMapping (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/aws-lambda/lib/function-base.ts:408:12)
at ManagedKafkaEventSource.bind (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/aws-lambda-event-sources/lib/kafka.ts:113:39)
at Function.addEventSource (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/aws-lambda/lib/function-base.ts:447:12)
at new Function (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/aws-lambda/lib/function.ts:824:12)
at new Guidewire (/Users/tobydickinson/repos/partner-integrations/cdk/lib/guidewire/Guidewire.ts:20:26)
at new PartnerIntegrationsStack (/Users/tobydickinson/repos/partner-integrations/cdk/lib/partner-integrations-stack.ts:125:5)
at Object. (/Users/tobydickinson/repos/partner-integrations/cdk/bin/partner-integrations.ts:7:1)
at Module._compile (node:internal/modules/cjs/loader:1103:14)
at Module.m._compile (/Users/tobydickinson/repos/partner-integrations/node_modules/ts-node/src/index.ts:1455:23)
at Module._extensions..js (node:internal/modules/cjs/loader:1155:10)
at Object.require.extensions. [as .ts] (/Users/tobydickinson/repos/partner-integrations/node_modules/ts-node/src/index.ts:1458:12)
at Module.load (node:internal/modules/cjs/loader:981:32)
at Function.Module._load (node:internal/modules/cjs/loader:822:12)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12)
at phase4 (/Users/tobydickinson/repos/partner-integrations/node_modules/ts-node/src/bin.ts:567:12)
at bootstrap (/Users/tobydickinson/repos/partner-integrations/node_modules/ts-node/src/bin.ts:85:10)
at main (/Users/tobydickinson/repos/partner-integrations/node_modules/ts-node/src/bin.ts:54:10)
at Object. (/Users/tobydickinson/repos/partner-integrations/node_modules/ts-node/src/bin.ts:717:3)
at Module._compile (node:internal/modules/cjs/loader:1103:14)
at Object.Module._extensions..js (node:internal/modules/cjs/loader:1155:10)
at Module.load (node:internal/modules/cjs/loader:981:32)
at Function.Module._load (node:internal/modules/cjs/loader:822:12)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12)
at node:internal/main/run_main_module:17:47
at Object.makeUniqueId (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/private/uniqueid.ts:42:11)
at PartnerIntegrationsStack.allocateLogicalId (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/stack.ts:975:12)
at PartnerIntegrationsStack.getLogicalId (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/stack.ts:473:28)
at CfnEventSourceMapping.synthesizeLogicalId (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/cfn-element.ts:131:25)
at Object.produce (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/cfn-element.ts:59:64)
at LazyString.resolve (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/lazy.ts:352:28)
at RememberingTokenResolver.resolveToken (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/resolvable.ts:150:24)
at RememberingTokenResolver.resolveToken (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/private/resolve.ts:283:18)
at resolve (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/private/resolve.ts:209:51)
at Object.resolve [as mapToken] (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/private/resolve.ts:116:77)
```
### Reproduction Steps
```ts
import { CfnMapping, Stack } from 'aws-cdk-lib';
import { Code, Function, Runtime, StartingPosition } from 'aws-cdk-lib/aws-lambda';
import { ManagedKafkaEventSource } from 'aws-cdk-lib/aws-lambda-event-sources';
import { Construct } from 'constructs';
export class MskErrorConstruct extends Construct {
constructor(scope: Stack, id: string, env: string) {
super(scope, id);
const cfnMapping = new CfnMapping(this, 'EnvironmentMapping', {
mapping: {
dev: {
topic: 'dev_mytopic_name'
},
staging: {
topic: 'staging_mytopic_name'
},
prod: {
topic: 'staging_mytopic_name'
}
}
})
new Function(this, 'MyFunction', {
runtime: Runtime.NODEJS_14_X,
code: Code.fromInline(`
export async function handler() {
return "hello world"
}
`),
handler: 'handler',
events: [ new ManagedKafkaEventSource({
clusterArn: 'myclusterarn',
startingPosition: StartingPosition.LATEST,
topic: cfnMapping.findInMap(env, 'topic'),
}),
],
});
}
}
```
### Possible Solution
_No response_
### Additional Information/Context
_No response_
### CDK CLI Version
2.43.1
### Framework Version
_No response_
### Node.js Version
16.14.0
### OS
MacOS Monterey 12.2.1
### Language
Typescript
### Language Version
_No response_
### Other information
_No response_
Contributor guide
Research direction
Start with packages/@aws-cdk/aws-lambda-event-sources/lib/kafka.ts, especially ManagedKafkaEventSource.bind, and compare it with the reproduction using CfnMapping.findInMap for the topic. The fix is complete when synthesis succeeds with a dynamic Kafka topic and no unresolved-token error is raised.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, typescript
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100