aws-cloudformation / aws-cloudformation/cloudformation-coverage-roadmap
[AWS::MSK::Cluster] - [Enhancement] - Propagate tags to service-managed ENIs / Network interfaces
- Dominant language
- No language data
- Stars
- 1.1k
- Forks
- 62
- PR merge metrics
- No merged PRs in 30d
Description
### Name of the resource
AWS::MSK::Cluster
### Resource name
_No response_
### Description
When an Amazon MSK (Managed Streaming for Apache Kafka) provisioned cluster is created, the service provisions ENIs in the account's VPC subnets for each broker node. These ENIs provide network connectivity for Kafka clients to communicate with the brokers. They do not inherit tags from the MSK cluster resource.
### Problem
- MSK creates one ENI per broker node in the specified client subnets. For example, a 3-broker cluster across 3 AZs creates at least 3 ENIs. Broker counts can be higher (e.g., 6, 9) depending on the configuration.
- These ENIs are requester-managed and carry no tags - no reference to the owning MSK cluster, broker ID, or Kafka version.
- ENIs can be replaced during broker scaling operations, version upgrades, or maintenance events.
- There is no mechanism -via CloudFormation or the MSK API- to control tags on these ENIs at creation time.
### Impact
Organizations enforcing mandatory tagging policies face:
- **Cost allocation gaps**: Untagged ENIs cannot be attributed to a project or team.
- **Compliance violations**: AWS Config rules like `required-tags` flag these ENIs as non-compliant.
- **Operational overhead**: Workarounds require event-driven Lambda functions to retroactively tag ENIs, with race conditions during broker scaling or maintenance events when ENIs are replaced.
- **Identification difficulty**: Without tags, correlating ENIs to a specific MSK cluster requires cross-referencing broker endpoints and private IP addresses via the `describe-cluster` or `list-nodes` API.
### Expected Behavior
Tags defined on `AWS::MSK::Cluster` should be automatically propagated to all ENIs created for broker nodes. When ENIs are replaced during lifecycle events (scaling, upgrades, maintenance), new ENIs should receive the current tags from the parent cluster.
### Other Details
- MSK requires subnets in at least 2 (Standard brokers) or 3 (Express brokers) different Availability Zones. Broker nodes are distributed evenly across the specified subnets.
- The same issue applies to `AWS::MSK::ServerlessCluster`, which also creates ENIs in the assigned subnets.
- Broker endpoints can be retrieved via `aws kafka get-bootstrap-brokers` or `aws kafka list-nodes`, but the corresponding ENIs must be correlated manually by private IP address.
### Reproduction
1. Create an MSK cluster with tags via CloudFormation:
```yaml
MyMSKCluster:
Type: AWS::MSK::Cluster
Properties:
ClusterName: my-kafka-cluster
KafkaVersion: "3.5.1"
NumberOfBrokerNodes: 3
BrokerNodeGroupInfo:
InstanceType: kafka.m5.large
ClientSubnets:
- subnet-aaa
- subnet-bbb
- subnet-ccc
SecurityGroups:
- sg-xxx
Tags:
Environment: production
CostCenter: "12345"
```
2. Retrieve broker node IPs:
```
aws kafka list-nodes --cluster-arn \
--query "NodeInfoList[].BrokerNodeInfo.Endpoints"
```
3. Find the corresponding ENIs by private IP:
```
aws ec2 describe-network-interfaces \
--filters "Name=addresses.private-ip-address,Values=" \
--query "NetworkInterfaces[].{ID:NetworkInterfaceId,Desc:Description,Tags:TagSet}"
```
4. Observe: `TagSet` is empty on the MSK-managed ENIs.
Contributor guide
Research direction
Start with the AWS::MSK::Cluster and AWS::MSK::ServerlessCluster coverage entries, then reproduce the behavior using the listed aws kafka list-nodes and aws ec2 describe-network-interfaces commands. Done means establishing whether tags from the parent resource can be propagated to MSK-managed ENIs and retained when those ENIs are replaced.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, kafka
- Domain
- cloud, networking
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100