aws-cloudformation / aws-cloudformation/cloudformation-coverage-roadmap

[AWS::MSK::Cluster] - [Enhancement] - Propagate tags to service-managed ENIs / Network interfaces

Open
#2,482 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
No language data
Stars
1.1k
Forks
62
PR merge metrics
No merged PRs in 30d

Description

### Name of the resource

AWS::MSK::Cluster

### Resource name

_No response_

### Description

When an Amazon MSK (Managed Streaming for Apache Kafka) provisioned cluster is created, the service provisions ENIs in the account's VPC subnets for each broker node. These ENIs provide network connectivity for Kafka clients to communicate with the brokers. They do not inherit tags from the MSK cluster resource.

### Problem

- MSK creates one ENI per broker node in the specified client subnets. For example, a 3-broker cluster across 3 AZs creates at least 3 ENIs. Broker counts can be higher (e.g., 6, 9) depending on the configuration.
- These ENIs are requester-managed and carry no tags - no reference to the owning MSK cluster, broker ID, or Kafka version.
- ENIs can be replaced during broker scaling operations, version upgrades, or maintenance events.
- There is no mechanism -via CloudFormation or the MSK API- to control tags on these ENIs at creation time.

### Impact

Organizations enforcing mandatory tagging policies face:
- **Cost allocation gaps**: Untagged ENIs cannot be attributed to a project or team.
- **Compliance violations**: AWS Config rules like `required-tags` flag these ENIs as non-compliant.
- **Operational overhead**: Workarounds require event-driven Lambda functions to retroactively tag ENIs, with race conditions during broker scaling or maintenance events when ENIs are replaced.
- **Identification difficulty**: Without tags, correlating ENIs to a specific MSK cluster requires cross-referencing broker endpoints and private IP addresses via the `describe-cluster` or `list-nodes` API.

### Expected Behavior

Tags defined on `AWS::MSK::Cluster` should be automatically propagated to all ENIs created for broker nodes. When ENIs are replaced during lifecycle events (scaling, upgrades, maintenance), new ENIs should receive the current tags from the parent cluster.

### Other Details

- MSK requires subnets in at least 2 (Standard brokers) or 3 (Express brokers) different Availability Zones. Broker nodes are distributed evenly across the specified subnets.
- The same issue applies to `AWS::MSK::ServerlessCluster`, which also creates ENIs in the assigned subnets.
- Broker endpoints can be retrieved via `aws kafka get-bootstrap-brokers` or `aws kafka list-nodes`, but the corresponding ENIs must be correlated manually by private IP address.

### Reproduction

1. Create an MSK cluster with tags via CloudFormation:
```yaml
MyMSKCluster:
Type: AWS::MSK::Cluster
Properties:
ClusterName: my-kafka-cluster
KafkaVersion: "3.5.1"
NumberOfBrokerNodes: 3
BrokerNodeGroupInfo:
InstanceType: kafka.m5.large
ClientSubnets:
- subnet-aaa
- subnet-bbb
- subnet-ccc
SecurityGroups:
- sg-xxx
Tags:
Environment: production
CostCenter: "12345"
```
2. Retrieve broker node IPs:
```
aws kafka list-nodes --cluster-arn \
--query "NodeInfoList[].BrokerNodeInfo.Endpoints"
```
3. Find the corresponding ENIs by private IP:
```
aws ec2 describe-network-interfaces \
--filters "Name=addresses.private-ip-address,Values=" \
--query "NetworkInterfaces[].{ID:NetworkInterfaceId,Desc:Description,Tags:TagSet}"
```
4. Observe: `TagSet` is empty on the MSK-managed ENIs.

Contributor guide

Open the contributing guide

Research direction

Start with the AWS::MSK::Cluster and AWS::MSK::ServerlessCluster coverage entries, then reproduce the behavior using the listed aws kafka list-nodes and aws ec2 describe-network-interfaces commands. Done means establishing whether tags from the parent resource can be propagated to MSK-managed ENIs and retained when those ENIs are replaced.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, kafka
Domain
cloud, networking
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.