aws-cloudformation / aws-cloudformation/cloudformation-coverage-roadmap
AWS::DMS::ReplicationTask - [BUG] - False-positive drift on ReplicationTaskSettings (service default enrichment / field duplication / log-component ordering)
- Dominant language
- No language data
- Stars
- 1.1k
- Forks
- 62
- PR merge metrics
- No merged PRs in 30d
Description
### Name of the resource
AWS::DMS::ReplicationTask
### Resource Name
_No response_
### Issue Description
Drift detection reports a false-positive drift on the `ReplicationTaskSettings` property of `AWS::DMS::ReplicationTask`. On create, DMS returns a settings document that differs from the template even when no value was changed: it adds default fields and null blocks that the template did not set, returns one control-table field under two different spellings at the same time, and reorders the `Logging.LogComponents` array while adding default components. Because the stored document can never match the template one-to-one, drift detection always reports the property as changed, although the effective configuration is identical.
### Expected Behavior
The stack stays `IN_SYNC` when the effective settings are unchanged. Drift detection should normalize `ReplicationTaskSettings` before comparing (ignore service-added defaults and null blocks, treat `LogComponents` as an unordered set, and handle the duplicated control-table field), so a task whose configuration matches the template is not reported as drift.
### Observed Behavior
Drift detection flags `/ReplicationTaskSettings` as `MODIFIED` on a freshly deployed stack, with no manual changes. A field-by-field comparison of the template value (Expected) against the value returned by DMS (Actual) shows:
- No user-set field differs in value. Every field present in the template has the same value in the live task (for example `FullLoadSettings.TransactionConsistencyTimeout` is `600` in both).
- The live document contains extra fields that the template does not set and that a user cannot control, for example `ChangeProcessingTuning.RecoveryTimeout`, and the null blocks `TTSettings`, `BeforeImageSettings`, `LoopbackPreventionSettings`, `CharacterSetSettings`, `PostProcessingRules`, plus `Logging.CloudWatchLogGroup`/`CloudWatchLogStream` set to null.
- `ControlTablesSettings` is returned with the same field under two spellings at once: `historyTimeslotInMinutes` and `HistoryTimeslotInMinutes`.
- `Logging.LogComponents` is returned in a different order, with default components added (ADDONS, COMMON, COMMUNICATION, DATA_STRUCTURE, FILE_FACTORY, FILE_TRANSFER). As a set, no component from the template is missing.
So the drift is caused entirely by service-side enrichment, field duplication, and array ordering, not by a real configuration change.
### Test Cases
This template is self-contained: it creates its own VPC and subnets, the smallest replication instance, and two dummy mysql endpoints with placeholder hosts (only created, never connected to). The task is never started and no database is involved. The `ReplicationTaskSettings` are provided as a fairly complete block using default values only, so nothing is intentionally different. Pick a `ReplicationInstanceClass` that `aws dms describe-orderable-replication-instances` returns in your region.
```yaml
AWSTemplateFormatVersion: '2010-09-09'
Description: Repro - false-positive drift on DMS ReplicationTaskSettings (service default enrichment)
Parameters:
ReplicationInstanceClass:
Type: String
Default: dms.t3.small
Resources:
Vpc:
Type: AWS::EC2::VPC
Properties:
CidrBlock: 10.0.0.0/16
Subnet1:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref Vpc
CidrBlock: 10.0.1.0/24
AvailabilityZone: !Select [0, !GetAZs '']
Subnet2:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref Vpc
CidrBlock: 10.0.2.0/24
AvailabilityZone: !Select [1, !GetAZs '']
ReplicationSubnetGroup:
Type: AWS::DMS::ReplicationSubnetGroup
Properties:
ReplicationSubnetGroupIdentifier: dms-drift-repro-settings-subnetgroup
ReplicationSubnetGroupDescription: Repro subnet group for settings drift
SubnetIds:
- !Ref Subnet1
- !Ref Subnet2
ReplicationInstance:
Type: AWS::DMS::ReplicationInstance
Properties:
ReplicationInstanceClass: !Ref ReplicationInstanceClass
ReplicationSubnetGroupIdentifier: !Ref ReplicationSubnetGroup
PubliclyAccessible: false
AllocatedStorage: 5
SourceEndpoint:
Type: AWS::DMS::Endpoint
Properties:
EndpointType: source
EngineName: mysql
ServerName: source.example.invalid
Port: 3306
Username: repro
Password: ReproPlaceholder123
TargetEndpoint:
Type: AWS::DMS::Endpoint
Properties:
EndpointType: target
EngineName: mysql
ServerName: target.example.invalid
Port: 3306
Username: repro
Password: ReproPlaceholder123
ReplicationTask:
Type: AWS::DMS::ReplicationTask
Properties:
ReplicationTaskIdentifier: dms-drift-repro-settings-01
MigrationType: full-load
ReplicationInstanceArn: !Ref ReplicationInstance
SourceEndpointArn: !Ref SourceEndpoint
TargetEndpointArn: !Ref TargetEndpoint
TableMappings: >-
{"rules":[{"rule-type":"selection","rule-id":"1","rule-name":"r1","object-locator":{"schema-name":"%","table-name":"%"},"rule-action":"include"}]}
ReplicationTaskSettings: >-
{"TargetMetadata":{"TargetSchema":"","SupportLobs":true,"FullLobMode":false,"LobChunkSize":64,"LimitedSizeLobMode":true,"LobMaxSize":32,"InlineLobMaxSize":0,"LoadMaxFileSize":0,"ParallelLoadThreads":0,"ParallelLoadBufferSize":0,"ParallelLoadQueuesPerThread":0,"ParallelApplyThreads":0,"ParallelApplyBufferSize":0,"ParallelApplyQueuesPerThread":0,"BatchApplyEnabled":false,"TaskRecoveryTableEnabled":false},"FullLoadSettings":{"TargetTablePrepMode":"DO_NOTHING","CreatePkAfterFullLoad":false,"StopTaskCachedChangesApplied":false,"StopTaskCachedChangesNotApplied":false,"MaxFullLoadSubTasks":8,"TransactionConsistencyTimeout":600,"CommitRate":10000},"Logging":{"EnableLogging":false,"EnableLogContext":false,"LogComponents":[{"Id":"TRANSFORMATION","Severity":"LOGGER_SEVERITY_DEFAULT"},{"Id":"SOURCE_UNLOAD","Severity":"LOGGER_SEVERITY_DEFAULT"},{"Id":"IO","Severity":"LOGGER_SEVERITY_DEFAULT"},{"Id":"TARGET_LOAD","Severity":"LOGGER_SEVERITY_DEFAULT"},{"Id":"PERFORMANCE","Severity":"LOGGER_SEVERITY_DEFAULT"},{"Id":"SOURCE_CAPTURE","Severity":"LOGGER_SEVERITY_DEFAULT"},{"Id":"SORTER","Severity":"LOGGER_SEVERITY_DEFAULT"},{"Id":"REST_SERVER","Severity":"LOGGER_SEVERITY_DEFAULT"},{"Id":"VALIDATOR_EXT","Severity":"LOGGER_SEVERITY_DEFAULT"},{"Id":"TARGET_APPLY","Severity":"LOGGER_SEVERITY_DEFAULT"},{"Id":"TASK_MANAGER","Severity":"LOGGER_SEVERITY_DEFAULT"},{"Id":"TABLES_MANAGER","Severity":"LOGGER_SEVERITY_DEFAULT"},{"Id":"METADATA_MANAGER","Severity":"LOGGER_SEVERITY_DEFAULT"}]},"ControlTablesSettings":{"ControlSchema":"","HistoryTimeslotInMinutes":5,"HistoryTableEnabled":false,"SuspendedTablesTableEnabled":false,"StatusTableEnabled":false,"FullLoadExceptionTableEnabled":false},"StreamBufferSettings":{"StreamBufferCount":3,"StreamBufferSizeInMB":8,"CtrlStreamBufferSizeInMB":5},"ChangeProcessingTuning":{"BatchApplyPreserveTransaction":true,"BatchApplyTimeoutMin":1,"BatchApplyTimeoutMax":30,"BatchApplyMemoryLimit":500,"BatchSplitSize":0,"MinTransactionSize":1000,"CommitTimeout":1,"MemoryLimitTotal":1024,"MemoryKeepTime":60,"StatementCacheSize":50},"ChangeProcessingDdlHandlingPolicy":{"HandleSourceTableDropped":true,"HandleSourceTableTruncated":true,"HandleSourceTableAltered":true},"ErrorBehavior":{"DataErrorPolicy":"LOG_ERROR","DataTruncationErrorPolicy":"LOG_ERROR","DataErrorEscalationPolicy":"SUSPEND_TABLE","DataErrorEscalationCount":0,"TableErrorPolicy":"SUSPEND_TABLE","TableErrorEscalationPolicy":"STOP_TASK","TableErrorEscalationCount":0,"RecoverableErrorCount":-1,"RecoverableErrorInterval":5,"RecoverableErrorThrottling":true,"RecoverableErrorThrottlingMax":1800,"RecoverableErrorStopRetryAfterThrottlingMax":true,"ApplyErrorDeletePolicy":"IGNORE_RECORD","ApplyErrorInsertPolicy":"LOG_ERROR","ApplyErrorUpdatePolicy":"LOG_ERROR","ApplyErrorEscalationPolicy":"LOG_ERROR","ApplyErrorEscalationCount":0,"ApplyErrorFailOnTruncationDdl":false,"FullLoadIgnoreConflicts":true,"FailOnTransactionConsistencyBreached":false,"FailOnNoTablesCaptured":true,"DataMaskingErrorPolicy":"STOP_TASK","EventErrorPolicy":"IGNORE"}}
```
Steps:
1. Deploy the stack and wait for `CREATE_COMPLETE` (the task takes a few minutes to reach the `ready` state).
2. Run drift detection (Console: Stack actions -> Detect drift, or CLI `detect-stack-drift`).
3. Look at the drift details for the ReplicationTask and compare Expected vs. Actual for `/ReplicationTaskSettings` field by field.
The stack goes to `DRIFTED` and `ReplicationTask` shows `MODIFIED` on `/ReplicationTaskSettings`, even though no user-set field differs in value.
### Other Details
Verified in `us-east-1` with a freshly deployed stack; no manual changes were made before running drift detection. The field lists above (added defaults, the duplicated `historyTimeslotInMinutes`/`HistoryTimeslotInMinutes`, and the six extra log components) are from this reproduction; the exact set of service-added fields may vary by DMS version. AWS Support case 178853082300312 tracks this internally.
Contributor guide
Research direction
Start with the supplied CloudFormation template and run `detect-stack-drift` after deployment, then compare the expected and actual `/ReplicationTaskSettings` values for `AWS::DMS::ReplicationTask`. Done means service-added defaults and null blocks, duplicated control-table fields, and `Logging.LogComponents` ordering no longer produce drift when effective settings are unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws
- Domain
- cloud, infrastructure
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100