aws / aws/aws-cdk

(lambda-event-sources): Unable to specify dynamic topic name in kafka

Open
#22,236 1 comment 0 reactions 0 assignees View on GitHub
@aws-cdk/aws-lambda-event-sources bug effort/small p2
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
2d 3h
Merged PRs (30d)
83

Description

### Describe the bug

A Kafka `kafkaTopic` with a dynamic name cannot be specified within a `ManagedKafkaEventSource`. An error is thrown when an unresolved token is provided.

This is because the ID for the event source mapping created during `bind` contains the topic name which means during synthesis, when the logical ID for the event source mapping is generated, a token is present in the ID for that construct which throws an error within the `uniqueid` [module](https://github.com/aws/aws-cdk/blob/7e424fdc45f98c83f0b19846c9683d7eeb767d80/packages/%40aws-cdk/core/lib/private/uniqueid.ts#L42)

### Expected Behavior

Dynamic topic name can be used and no error should be thrown during sythesis

### Current Behavior

Error thrown during synthesis

```
Error: Resolution error: ID components may not include unresolved tokens: KafkaEventSource:PartnerIntegrationsStacklocalGuidewireGuidewireSubroReportSenderEFF9AC27${Token[TOKEN.304]}.
Object creation stack:
at Function.uncachedString (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/lazy.ts:187:27)
at new CfnElement (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/cfn-element.ts:59:27)
at new CfnRefElement (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/cfn-element.ts:146:1)
at new CfnResource (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/cfn-resource.ts:85:5)
at new CfnEventSourceMapping (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/aws-lambda/lib/lambda.generated.ts:1778:9)
at new EventSourceMapping (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/aws-lambda/lib/event-source-mapping.ts:316:35)
at Function.addEventSourceMapping (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/aws-lambda/lib/function-base.ts:408:12)
at ManagedKafkaEventSource.bind (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/aws-lambda-event-sources/lib/kafka.ts:113:39)
at Function.addEventSource (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/aws-lambda/lib/function-base.ts:447:12)
at new Function (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/aws-lambda/lib/function.ts:824:12)
at new Guidewire (/Users/tobydickinson/repos/partner-integrations/cdk/lib/guidewire/Guidewire.ts:20:26)
at new PartnerIntegrationsStack (/Users/tobydickinson/repos/partner-integrations/cdk/lib/partner-integrations-stack.ts:125:5)
at Object. (/Users/tobydickinson/repos/partner-integrations/cdk/bin/partner-integrations.ts:7:1)
at Module._compile (node:internal/modules/cjs/loader:1103:14)
at Module.m._compile (/Users/tobydickinson/repos/partner-integrations/node_modules/ts-node/src/index.ts:1455:23)
at Module._extensions..js (node:internal/modules/cjs/loader:1155:10)
at Object.require.extensions. [as .ts] (/Users/tobydickinson/repos/partner-integrations/node_modules/ts-node/src/index.ts:1458:12)
at Module.load (node:internal/modules/cjs/loader:981:32)
at Function.Module._load (node:internal/modules/cjs/loader:822:12)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12)
at phase4 (/Users/tobydickinson/repos/partner-integrations/node_modules/ts-node/src/bin.ts:567:12)
at bootstrap (/Users/tobydickinson/repos/partner-integrations/node_modules/ts-node/src/bin.ts:85:10)
at main (/Users/tobydickinson/repos/partner-integrations/node_modules/ts-node/src/bin.ts:54:10)
at Object. (/Users/tobydickinson/repos/partner-integrations/node_modules/ts-node/src/bin.ts:717:3)
at Module._compile (node:internal/modules/cjs/loader:1103:14)
at Object.Module._extensions..js (node:internal/modules/cjs/loader:1155:10)
at Module.load (node:internal/modules/cjs/loader:981:32)
at Function.Module._load (node:internal/modules/cjs/loader:822:12)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12)
at node:internal/main/run_main_module:17:47
at Object.makeUniqueId (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/private/uniqueid.ts:42:11)
at PartnerIntegrationsStack.allocateLogicalId (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/stack.ts:975:12)
at PartnerIntegrationsStack.getLogicalId (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/stack.ts:473:28)
at CfnEventSourceMapping.synthesizeLogicalId (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/cfn-element.ts:131:25)
at Object.produce (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/cfn-element.ts:59:64)
at LazyString.resolve (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/lazy.ts:352:28)
at RememberingTokenResolver.resolveToken (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/resolvable.ts:150:24)
at RememberingTokenResolver.resolveToken (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/private/resolve.ts:283:18)
at resolve (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/private/resolve.ts:209:51)
at Object.resolve [as mapToken] (/Users/tobydickinson/repos/partner-integrations/node_modules/aws-cdk-lib/core/lib/private/resolve.ts:116:77)
```

### Reproduction Steps

```ts
import { CfnMapping, Stack } from 'aws-cdk-lib';
import { Code, Function, Runtime, StartingPosition } from 'aws-cdk-lib/aws-lambda';
import { ManagedKafkaEventSource } from 'aws-cdk-lib/aws-lambda-event-sources';
import { Construct } from 'constructs';

export class MskErrorConstruct extends Construct {
constructor(scope: Stack, id: string, env: string) {
super(scope, id);

const cfnMapping = new CfnMapping(this, 'EnvironmentMapping', {
mapping: {
dev: {
topic: 'dev_mytopic_name'
},
staging: {
topic: 'staging_mytopic_name'
},
prod: {
topic: 'staging_mytopic_name'
}
}
})

new Function(this, 'MyFunction', {
runtime: Runtime.NODEJS_14_X,
code: Code.fromInline(`
export async function handler() {
return "hello world"
}
`),
handler: 'handler',
events: [ new ManagedKafkaEventSource({
clusterArn: 'myclusterarn',
startingPosition: StartingPosition.LATEST,
topic: cfnMapping.findInMap(env, 'topic'),
}),
],
});
}
}

```

### Possible Solution

_No response_

### Additional Information/Context

_No response_

### CDK CLI Version

2.43.1

### Framework Version

_No response_

### Node.js Version

16.14.0

### OS

MacOS Monterey 12.2.1

### Language

Typescript

### Language Version

_No response_

### Other information

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with packages/@aws-cdk/aws-lambda-event-sources/lib/kafka.ts, especially ManagedKafkaEventSource.bind, and compare it with the reproduction using CfnMapping.findInMap for the topic. The fix is complete when synthesis succeeds with a dynamic Kafka topic and no unresolved-token error is raised.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, typescript
Domain
cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.