aws-cloudformation / aws-cloudformation/cloudformation-coverage-roadmap
AWS::KinesisFirehose::DeliveryStream - BUG - ElasticsearchDestinationConfiguration.ProcessingConfiguration
- Dominant language
- No language data
- Stars
- 1.1k
- Forks
- 62
- PR merge metrics
- No merged PRs in 30d
Description
## Scope of request
Changes made to the `ElasticsearchDestinationConfiguration.ProcessingConfiguration` as part of a DeliveryStream are not reflected. This can be confirmed through CloudTrail logs. The `processingConfiguration` object is present in the API call from CloudFormation during creation but not during update. Current workaround is to destroy and recreate the Firehose Delivery Stream (which is not acceptable in all cases). Other workaround is to manually make configuration changes through the console, which leads to changes which are not tracked through infrastructure as code templates.
## Expected behavior
Updates made to the `ElasticsearchDestinationConfiguration.ProcessingConfiguration` attribute are correctly reflected during a stack update.
## Steps to Reproduce
Create a stack with the following Delivery Stream (or similar):
```
FirehoseDeliveryStream:
Type: AWS::KinesisFirehose::DeliveryStream
Properties:
ElasticsearchDestinationConfiguration:
BufferingHints:
IntervalInSeconds: 300
SizeInMBs: 50
DomainARN: !GetAtt ElasticsearchDomain.Arn
IndexName: example-index
IndexRotationPeriod: OneDay
RetryOptions:
DurationInSeconds: 900
RoleARN: !GetAtt DeliveryStreamRole.Arn
S3BackupMode: FailedDocumentsOnly
S3Configuration:
BucketARN: !GetAtt Bucket.Arn
BufferingHints:
IntervalInSeconds: 300
SizeInMBs: 50
CompressionFormat: GZIP
RoleARN: !GetAtt DeliveryStreamRole.Arn
TypeName: ""
```
Once the stack has been created successfully, update the stack with the following (or similar):
```
FirehoseDeliveryStream:
Type: AWS::KinesisFirehose::DeliveryStream
Properties:
ElasticsearchDestinationConfiguration:
BufferingHints:
IntervalInSeconds: 300
SizeInMBs: 50
DomainARN: !GetAtt ElasticsearchDomain.Arn
IndexName: example-index
IndexRotationPeriod: OneDay
ProcessingConfiguration:
Enabled: true
Processors:
- Type: Lambda
Parameters:
- ParameterName: LambdaArn
ParameterValue: !GetAtt FirehoseDataTransformationFunction.Arn
- ParameterName: RoleArn
ParameterValue: !GetAtt DeliveryStreamRole.Arn
RetryOptions:
DurationInSeconds: 900
RoleARN: !GetAtt DeliveryStreamRole.Arn
S3BackupMode: FailedDocumentsOnly
S3Configuration:
BucketARN: !GetAtt Bucket.Arn
BufferingHints:
IntervalInSeconds: 300
SizeInMBs: 50
CompressionFormat: GZIP
RoleARN: !GetAtt DeliveryStreamRole.Arn
TypeName: ""
```
The ProcessingConfiguration change will not be reflected in the delivery stream, despite the fact that CloudFormation detects that the resource has change and the events will show that it was updated successfully. This can be confirmed by making a call to `DescribeDeliveryStream` or by viewing the configuration of the resource through the management console.
## Category
#### Analytics
Contributor guide
Assessment
This issue has not been assessed yet.