awslabs / awslabs/aws-deployment-framework
[Bug]: Issue installing adf in new account
- Dominant language
- Python
- Stars
- 699
- Forks
- 235
- Avg merge
- 20h 53m
- Merged PRs (30d)
- 7
Description
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Describe the bug
I just setup a new management account with Control Tower configured.
When installing ADF v.4.0.0 in this account with the following flags using a Cloud9 environment
`{"CrossAccountAccessRoleName": "AWSControlTowerExecution", "MainNotificationEndpoint": "", "DeploymentAccountName": "", "DeploymentAccountEmailAddress": "", "DeploymentAccountAlias": "", "DeploymentAccountId": "ID-OF-DEPLOY-ACC", "DeploymentAccountMainRegion": "eu-west-1", "DeploymentAccountTargetRegions": "us-east-1", "ProtectedOUs": "", "LogLevel": "INFO", "AllowBootstrappingOfManagementAccount": "No", "GrantOrgWidePrivilegedBootstrapAccessUntil": "1900-12-31T23:59:59Z"}`
the installation successfully completes but the aws-deployment-framework-bootstrap-pipeline fails with
```
[Container] 2024/07/12 14:28:55.222874 Running command python adf-build/store_config.py
--
494 | 2024-07-12 14:28:55,479 \| INFO \| __main__ \| ADF Version v4.0.0 \| (store_config.py:25)
495 | 2024-07-12 14:28:55,479 \| INFO \| __main__ \| ADF Log Level is INFO \| (store_config.py:26)
496 | 2024-07-12 14:28:55,479 \| INFO \| __main__ \| Storing configuration values in AWS Systems Manager Parameter Store. \| (store_config.py:28)
497 | 2024-07-12 14:28:55,562 \| INFO \| config \| Using default ADF config: ./adfconfig.yml \| (config.py:115)
498 | 2024-07-12 14:28:57,198 \| INFO \| __main__ \| Configuration values stored successfully. \| (store_config.py:33)
499 |
500 | [Container] 2024/07/12 14:28:57.267522 Running command aws s3 sync ./adf-build/shared s3://$SHARED_MODULES_BUCKET/adf-build --only-show-errors
501 |
502 | [Container] 2024/07/12 14:29:13.464580 Running command aws s3 sync . s3://$S3_BUCKET --only-show-errors --delete
503 |
504 | [Container] 2024/07/12 14:31:16.067325 Running command python adf-build/shared/helpers/sync_to_s3.py --extension .yml --extension .yaml --metadata adf_version=${ADF_VERSION} --upload-with-metadata execution_id=${CODEPIPELINE_EXECUTION_ID} --recursive adf-accounts s3://$ACCOUNT_BUCKET
505 | INFO:__main__:All done.
506 |
507 | [Container] 2024/07/12 14:31:16.636245 Running command sleep 10
508 |
509 | [Container] 2024/07/12 14:31:26.642540 Running command python adf-build/main.py
510 | 2024-07-12 14:31:26,956 \| INFO \| __main__ \| ADF Version v4.0.0 \| (main.py:464)
511 | 2024-07-12 14:31:26,957 \| INFO \| __main__ \| ADF Log Level is INFO \| (main.py:465)
512 | 2024-07-12 14:31:27,423 \| INFO \| config \| Using default ADF config: ./adfconfig.yml \| (config.py:115)
513 | 2024-07-12 14:31:27,501 \| WARNING \| organizations \| Deprecation warning: Using a role in the organizations client is being deprecated. Please provide the relevant clients to remove this warning \| (organizations.py:44)
514 | 2024-07-12 14:31:27,711 \| INFO \| organization_policy \| Determining if Organization Policy changes are required. (Tagging or Service Controls) \| (organization_policy.py:123)
515 | 2024-07-12 14:31:41,227 \| INFO \| organizations \| SERVICE_CONTROL_POLICY are currently enabled within the Organization \| (organizations.py:111)
516 | 2024-07-12 14:31:42,821 \| INFO \| organizations \| TAG_POLICY are currently enabled within the Organization \| (organizations.py:111)
517 | 2024-07-12 14:31:42,921 \| INFO \| sts \| Using ADF Account-Bootstrapping Jump Role to assume into account REDACTED \| (sts.py:83)
518 | 2024-07-12 14:31:42,999 \| INFO \| sts \| Assumed into arn:aws:iam::REDACTED:role/adf/account-bootstrapping/jump/adf-bootstrapping-cross-account-jump-role with session name: management \| (sts.py:42)
519 | 2024-07-12 14:31:43,137 \| INFO \| sts \| Assumed into arn:aws:iam::REDACTED:role/AWSControlTowerExecution with session name: management \| (sts.py:42)
520 | 2024-07-12 14:31:43,147 \| WARNING \| sts \| Using the privileged cross-account access role: AWSControlTowerExecution, as access to this role was granted for account REDACTED \| (sts.py:107)
521 | Traceback (most recent call last):
522 | File "/codebuild/output/src3016576730/src/adf-build/main.py", line 577, in
523 | main()
524 | File "/codebuild/output/src3016576730/src/adf-build/main.py", line 483, in main
525 | deployment_account_role = prepare_deployment_account(
526 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^
527 | File "/codebuild/output/src3016576730/src/adf-build/main.py", line 205, in prepare_deployment_account
528 | if '@' not in config.notification_endpoint:
529 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
530 | TypeError: argument of type 'NoneType' is not iterable
531 |
532 | [Container] 2024/07/12 14:31:45.939501 Command did not exit successfully python adf-build/main.py exit status 1
533 | [Container] 2024/07/12 14:31:45.943328 Phase complete: BUILD State: FAILED
534 | [Container] 2024/07/12 14:31:45.943350 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: python adf-build/main.py. Reason: exit status 1
535 | [Container] 2024/07/12 14:31:46.019122 Entering phase POST_BUILD
536 | [Container] 2024/07/12 14:31:46.022013 Phase complete: POST_BUILD State: SUCCEEDED
537 | [Container] 2024/07/12 14:31:46.022026 Phase context status code: Message:
```
I assume it's attempting to send a notification but is unable to do so as I set "MainNotificationEndpoint": ""
I would like some help in figuring out if the notification error is the "real" error or if this error is just hiding some other underlying issue.
### Expected Behavior
If I set "MainNotificationEndpoint": "" it won't try to run the notification part of the code
### Current Behavior
Setting "MainNotificationEndpoint": "" results in an error
### Steps To Reproduce
1. Install ADF v4 with these flags
`{"CrossAccountAccessRoleName": "AWSControlTowerExecution", "MainNotificationEndpoint": "", "DeploymentAccountName": "", "DeploymentAccountEmailAddress": "", "DeploymentAccountAlias": "", "DeploymentAccountId": "ID-OF-DEPLOY-ACC", "DeploymentAccountMainRegion": "eu-west-1", "DeploymentAccountTargetRegions": "us-east-1", "ProtectedOUs": "", "LogLevel": "INFO", "AllowBootstrappingOfManagementAccount": "No", "GrantOrgWidePrivilegedBootstrapAccessUntil": "1900-12-31T23:59:59Z"}`
2. The aws-deployment-framework-bootstrap-pipeline pipeline step UploadAndUpdateBaseStacks will fail
### Possible Solution
Workaround is to modify the adfconfig.yml in codecommit and add a notification endpoint. Once added the pipeline will succeed.
### Additional Information/Context
_No response_
### ADF Version
v4.0.0
### Contributing a fix?
- [ ] Yes, I am working on a fix to resolve this issue
Contributor guide
Research direction
Start with adf-build/main.py around prepare_deployment_account and the reported notification_endpoint check, then review how the value is loaded in config.py from adfconfig.yml. Reproduce the v4.0.0 UploadAndUpdateBaseStacks failure with an empty MainNotificationEndpoint. Done means an empty endpoint no longer causes the pipeline to fail, while the existing configured-endpoint path remains functional.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud, devops
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 38/100