aws-cloudformation / aws-cloudformation/cloudformation-coverage-roadmap

AWS::MSK::Cluster - Ignore VolumeSize during updates if AutoScaling is enabled.

Open
#1,129 0 comments 2 reactions 0 assignees View on GitHub
enhancement
Dominant language
No language data
Stars
1.1k
Forks
62
PR merge metrics
No merged PRs in 30d

Description

### Name of the resource

AWS::MSK::Cluster

### Resource name

AWS::MSK::Cluster

### Description

**Issue Description**
The MSK Cluster is being created by CloudFormation. The initial volume size is being specified using this property AWS::MSK::Cluster EBSStorageInfo [VolumeSize](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-msk-cluster-ebsstorageinfo.html#cfn-msk-cluster-ebsstorageinfo-volumesize).

Because of the feature to automatically expand storage [here](https://docs.aws.amazon.com/msk/latest/developerguide/msk-autoexpand.html), the actual volume size on the node increases, but the value in CloudFormation remains the same (expected).

But when an update is made to any of the Properties under `EBSStorageInfo` Property , the volumes sizes are not the same. CloudFormation tries to update cluster broker nodes with the initial value, but this fails with the error:
```
Resource handler returned message: "[ClientRequestToken: xxxxxxxxxxxx To update storage, you must increase it by at least 10 GiB. (Service: Kafka, Status Code: 400, Request ID: xxxxxxxxxxxx, Extended Request ID: null) 'volumeSizeGB'"
```

**Expected Behavior**

The changes made to the VolumeSize property by Auto expand feature should be ignored during Stack Updates.

**Current Workaround**

1. Modify the VolumeSize property in CloudFormation template to match the Volume Size from the MSK Cluster Console.
2. Do not specify VolumeSize property in the template and manage it out-of-band i.e. will not cause issues during future updates.

### Other Details

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with the AWS::MSK::Cluster resource and its EBSStorageInfo.VolumeSize update behavior, then review the linked CloudFormation and MSK auto-expansion documentation. Reproduce a stack update after MSK storage has auto-expanded; done means updates to other EBSStorageInfo properties no longer reuse the stale VolumeSize or fail the minimum-increase validation.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, kafka
Domain
cloud, infrastructure
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.