aws / aws/chalice

Add support for MSK event source

Open
#1,721 3 comments 11 reactions 0 assignees View on GitHub
feature-request
Dominant language
Python
Stars
11.1k
Forks
1k
Avg merge
1d 22h
Merged PRs (30d)
2

Description

I'd like to have MSK as an event source. Currently I work around this by adding the required configuration during packaging with the `--merge-template` option:

```
{
"Resources": {
"Handler": {
"Properties": {
"Events": {
"KafkaConsumer": {
"Type": "MSK",
"Properties": {
"StartingPosition": "LATEST",
"Stream": "arn",
"Topics": [
"topic1"
]
}
}
},
"VpcConfig": {
"SubnetIds": [
"subnet"
],
"SecurityGroupIds": [
"security-group-id"
]
}
}
}
}
}
```

The SAM should contain

* An `Event` with type `"MSK"`
* A `VpcConfig` property with subnets and security groups

The `Event` could follow the same API as the other sources, but I wouldn't how to nicely configure the vpc settings. Its my understanding that its required deploy the function in the same VPC as the MSK cluster for the MSK trigger to work. Furthermore, the VPC needs to have VPC endpoints to the STS and Lambda services but that's not necessarily what Chalice should do.

I'm open to contributing myself towards this functionality. I've already forked the repo and will try to create a PR. Input and discussion is more than welcome.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing Chalice's existing event-source APIs and the current --merge-template packaging path described in the issue. The work is done when MSK events are supported with the requested event configuration and VpcConfig for subnets and security groups, with coverage for the generated configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.